Real-time Model Inference in a Video Streaming Environment //Jacob Tsafatinos // MLOps Coffee Sessions #97

MLOps Coffee Sessions #97 with Jacob Tsafatinos, Real-time Model Inference in a Video Streaming Environment co-hosted by Adam Sroka.

// Abstract
A few years ago Uber set out to create an ads platform for the Uber Eats app that relied heavily on three pillars; Speed, Reliability, and Accuracy. Some of the technical challenges they were faced with included exactly-once semantics in real-time. To accomplish this goal, they created the architecture diagram above with lots of love from Flink, Kafka, Hive, and Pinot. You can dig into the whole paper (https://go.mlops.community/k8gzZd) to see all the reasoning for their design decisions.

// Bio
Jacob Tsafatinos is a Staff Software Engineer at Elemy. He led the efforts of the Ad Events Processing system at Uber and has previously worked on a range of problems including data ingestion for search and machine learning recommendation pipelines. In his spare time, he can be found playing lead guitar in his band Good Kid.

// MLOps Jobs board  
https://mlops.pallet.xyz/jobs

// Related Links
Uber blog
https://eng.uber.com/author/jacob-tsafatinos/
https://eng.uber.com/real-time-exactly-once-ad-event-processing/

--------------- ✌️Connect With Us ✌️ -------------
Join our slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/

Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Mihail on LinkedIn: https://www.linkedin.com/in/mihaileric/
Connect with Jacob on LinkedIn: https://www.linkedin.com/in/jacobtsaf/

Timestamps:
[00:00] Introduction to Jacob Tsafatinos
[00:40] Takeaways
[04:25] Jacob's band
[05:29] Lyrics about software engineers or artistic stuff
[06:20] Connection of hobby and real-time system
[08:43] How to game Spotify Algorithm?
[10:00] Data stack for analytics
[13:28] Uber blog
[16:28] Video mess up
[17:04] Considerations and importance of the Uber System
[21:22] Challenges encountered through the Uber System journey
[26:06] Crucial to building the system
[28:13] Not exactly real-time
[30:22] Design decisions main questions
[34:23] Testament to OSS  
[36:58] Real-time processing systems for analytical use cases vs Real-time processing systems for predictive use cases
[38:46] Real-time systems necessity
[41:04] Potential that opens up new doors
[41:40] Runaway or learn it?
[46:09] Real-time use case target
[49:31] Resource constrained
[50:48] ML Oops stories
[52:45] Wrap up

Episode source