Abnormal Security identifies and blocks advanced social engineering attacks in an ever-changing threat landscape, and so rapid feature development is of paramount importance for staying ahead of attackers. As we've scaled our machine learning system to serve thousands of features for hundreds of machine learning models, it's become a major focus to balance stability with rapid iteration speed. Last year at apply(), we saw a great talk from Stitch Fix on their Hamilton framework for managing complicated logic in Pandas Dataframes. In this talk, we present a similar framework called Compass that was developed at Abnormal. Compass takes a similar approach to Hamilton by modeling feature extraction pipelines as a DAG of composable functions, but differs in some key design choices that make it a better fit for Abnormal's ML use-case and tech stack. We'll show how Compass enables machine learning engineers to express feature extraction logic in simple, pipeline-agnostic Python functions, while also providing a way to interface with a feature store in a scalable way when needed.
