As companies become increasingly data driven, the technologies underlying these rich insights have grown more nuanced and complex. While our ability to collect, store, aggregate, and visualize this data has largely kept up with the needs of modern data and ML teams, the mechanics behind data quality and integrity has lagged. To keep pace with data’s clock speed of innovation, data engineers need to invest not only in the latest modeling and analytics tools, but also ML-based technologies that can increase data accuracy and prevent broken pipelines. The solution? Data observability, the next frontier of data engineering. I'll discuss why data observability matters to building a better data quality strategy and tactics best-in-class organizations use to address it -- including org structure, culture, and technology.
