Tecton
timezone
+00:00 GMT
SIGN IN
  • Home
  • Events
  • Content
  • Help
Sign In
Sign in or Join the community to continue

Streaming is just an implementation detail

Posted Apr 12
# Data engineering
# Systems and Architecture
Share
SPEAKER
Arjun Narayan
Arjun Narayan
Arjun Narayan
CEO / Co-Founder, Materialize @ Materialize

Arjun Narayan is the co-founder and CEO of Materialize. Materialize is a streaming database for real-time applications and analytics, built on top of a next generation stream processor - Timely Dataflow. Arjun was previously an engineer at Cockroach Labs and holds a Ph.D in Computer Science from the University of Pennsylvania.

+ Read More

Arjun Narayan is the co-founder and CEO of Materialize. Materialize is a streaming database for real-time applications and analytics, built on top of a next generation stream processor - Timely Dataflow. Arjun was previously an engineer at Cockroach Labs and holds a Ph.D in Computer Science from the University of Pennsylvania.

+ Read More
SUMMARY

Microservices are stream processing; whether you're using Redis, Kafka, or gRPC, you continuously handle events and manage consistency. And given that these are some of the most challenging problems in databases, you're probably not doing a very good job at it.

But that's not your fault, these problems are hard! Just like you wouldn't implement your own database for every web service, you shouldn't be building your own stream processor for every new product feature.

Today's stream processors have failed to gain widespread adoption outside niche use cases because they put themselves at the forefront. They force you to think about streaming when building your application and when deploying to production.

In my talk, I will argue that "streaming" is the right tool but the wrong abstraction. Of course, we all want the benefits - increased speed, stronger consistency - but they need to meet developers where they are. Only when streaming becomes an implementation detail can it gain widespread adoption and bring forth the benefits it has promised for so long.

+ Read More

Watch More

10
Posted Jul 14 | Views 22
# Data engineering
# Feature Stores
# Production Use Case
# Systems and Architecture