Tecton
timezone
+00:00 GMT
SIGN IN
  • Home
  • Events
  • Content
  • Help
Sign In
Sign in or Join the community to continue

Model Calibration in the Etsy Ads Marketplace

Posted Jan 04
# Explainability and Observability
# Organization and Processes
# Production Use Case
Share
SPEAKERS
Erica Greene
Erica Greene
Erica Greene
Senior Engineering Manager @ Etsy

Erica Greene is an engineering manager based in Brooklyn, NY. She loves working with teams to design and build ML-driven products.

+ Read More

Erica Greene is an engineering manager based in Brooklyn, NY. She loves working with teams to design and build ML-driven products.

+ Read More
Seoyoon Park
Seoyoon Park
Seoyoon Park
Senior Software Engineer @ Etsy

Seoyoon is a Senior Software Engineer on the Ad Ranking team at Etsy. He has worked on various projects which includes adding tooling to introspect data quality, assessing model performance, and most recently calibrating ML models used for ranking. Prior to Etsy, he was at Signafire, a data analytics startup, where he built pipelines for data visualization and analysis.

+ Read More

Seoyoon is a Senior Software Engineer on the Ad Ranking team at Etsy. He has worked on various projects which includes adding tooling to introspect data quality, assessing model performance, and most recently calibrating ML models used for ranking. Prior to Etsy, he was at Signafire, a data analytics startup, where he built pipelines for data visualization and analysis.

+ Read More
SUMMARY

When displaying relevant first-party ads to buyers in the Etsy marketplace, ads are ranked using a combination of outputs from ML models. The relevance of ads displayed to buyers and costs charged to sellers are highly sensitive to the output distributions of the models. Various factors contribute to model outputs which include the makeup of training data, model architecture, and input features. To make the system more robust and resilient to modeling changes, we have calibrated all ML models that power ranking and bidding.

In this talk, we will first discuss the pain points and use cases that identified the need for calibration in our system. We will share the journey, learnings, and challenges of calibrating our machine learning models and the implications of calibrated outputs. Finally, we will explain how we are using the calibrated outputs in downstream applications and explore opportunities that calibration unlocks at Etsy.

+ Read More

Watch More

10
Posted May 12 | Views 105
# Feature Stores
# Production Use Case
# Systems and Architecture
10
Posted Apr 12 | Views 17
# Model serving
# Organization and Processes
# Production Use Case
# Systems and Architecture
10
Posted Apr 12 | Views 18
# Model serving
# Model training
# Production Use Case
# Systems and Architecture