Alexander Gallego is the founder and CEO of Vectorized. Vectorized offers a product called RedPanda, an Apache Kafka-compatible event streaming platform that’s significantly faster and easier to operate than Kafka. We talk about the increasing ubiquity of streaming platforms, what they’re used for, why Kafka is slow, and how to safely and effectively build a replacement.

Previously, Alex was a Principal Software Engineer at Akamai systems and the creator of the Concord Framework, a distributed stream processing engine built in C++ on top of Apache Mesos.

Apple Podcasts | Spotify | Google Podcasts

Subscribe now

Share Software at Scale

Highlights

7:00 - Who uses streaming platforms, and why? Why would someone use Kafka?

12:30 - What would be the reason to use Kafka over Amazon SQS or Google PubSub?

17:00 - What makes Kafka slow? The story behind RedPanda. We talk about memory efficiency in RedPanda which is better optimized for machines with more cores.

34:00 - Other optimizations in RedPanda

39:00 - WASM programming within the streaming engine, almost as if Kafka was an AWS Lambda processor.

43:00 - How to convince potential customers to switch from Kafka to Redpanda?

48:00 - What is the release process for Redpanda? How do they ensure that a new version isn’t broken?

52:00 - What have we learnt about the state of Kafka and the use of streaming tools?

Episode source