Data

Kafka is Not Your Message Bus—Here’s Why

Kafka is awesome. Its unique approach to log aggregation and messaging allows it to scale to Gbs/s while persisting every message (hence the “log aggregation” part). But, like everything in engineering, building Kafka required a series of deliberate trade-offs. The choices made by the LinkedIn team were pivotal in creating something fundamentally different from traditional messaging systems like RabbitMQ. Had they chosen differently, Kafka might look much closer to RabbitMQ than it does today.

Read more