Wild Wild Tests: Monitoring Recommender Systems in the Wild

Posted Apr 12, 2022 | Views 714

# Explainability and Observability

# Open Source

# Production Use Case

SPEAKERS

Jacopo Tagliabue

Director of AI @ Coveo

Educated in several acronyms across the globe (UNISR, SFI, MIT), Jacopo Tagliabue was co-founder of Tooso, an A.I. company acquired by Coveo in 2019. Jacopo is currently the Director of A.I. at Coveo, shipping models to hundreds of customers and millions of users. When not busy building products, he teaches MLSys at NYU and explores topics at the intersection of language, reasoning and learning (with research work presented at NAACL, RecSys, ACL, SIGIR). In previous lives, he managed to get a Ph.D., do sciency things for a pro basketball team, and simulate a pre-Columbian civilization.

+ Read More

Federico Bianchi

Postdoctoral Researcher @ Bocconi University

+ Read More

SUMMARY

As with most Machine Learning systems, recommender systems are typically evaluated through performance metrics computed over held-out data points. However, real-world behavior is undoubtedly nuanced, and case-specific tests must be employed to ensure the desired quality. We introduce RecList, a behavioral-based testing methodology and open source package for RecSys, designed to scale up testing through sensible defaults, extensible abstractions and wrappers for popular datasets.

+ Read More

Watch More

44:35

apply(recsys) Conference 2022 | Lessons Learned: The Journey to Operationalizing Recommender Systems

Posted Dec 12, 2022 | Views 651

# apply(recsys) 2022

# Panel Discussion

# Production Use Case

37:05

apply(recsys) Conference 2022 | Crawl, walk, run: a practical intro to applied recommender systems

Posted Dec 12, 2022 | Views 645

# apply(recsys) 2022

# Systems and Architecture

# Production Use Case

Building Malleable ML Systems through Measurement, Monitoring & Maintenance

Posted Jul 14, 2021 | Views 357

# Explainability and Observability

# Organization and Processes

# Research