I'm trying to understand when it's better to use a Pub/Sub system compared to a Kafka topic for message broadcasting. They seem to operate on similar principles of sending messages to various subscribers or services. What criteria should I consider when deciding between the two?
1 Answer
I've recently had to make this decision at work while migrating some old systems. I found that Pub/Sub is fantastic for simple, fire-and-forget messaging where you don't need to stress over message order or persistence, like sending out notifications or triggering webhooks. On the other hand, Kafka is unbeatable when message durability, ordering, or the ability to replay events is crucial. For our notifications, we decided to go with Google Pub/Sub because it was easier to set up, but we kept Kafka for our main event sourcing pipeline due to its durability and ordering guarantees. Just a heads up, Kafka does require more maintenance, but it gives you much greater control over your data flow.
Thanks for the insight! So when you mentioned keeping Kafka for event sourcing, does that mean you integrated it with your notification system for durability and ordering too?