I'm working on a system that may ingest a massive amount of data. A Premium Event Hubs namespace with 16 processing units and 100 partitions may not provide enough capacity. Could I place an Azure Load Balancer in front of three Premium Event Hubs namespaces and distribute producers across them? The clients use TCP connections on port 9093, so I'm also wondering whether that creates any issues. Are there ingress-rate limits or other Azure Load Balancer constraints that would prevent this design?
2 Answers
A load balancer won’t provide a transparent way to combine several Event Hubs namespaces. If you stay with Premium, the usual design is to shard producers explicitly across separate namespaces and make sure consumers know which namespace contains each stream. Benchmark that setup against Event Hubs Dedicated, since the right choice depends on your throughput, partitioning, latency, and operational requirements.
An Azure Load Balancer probably isn’t suitable for this. TCP port 9093 isn’t the main issue; Event Hubs namespaces or their private endpoints can’t be used as backend targets in a Load Balancer pool. Azure Load Balancer is primarily designed for IaaS resources such as virtual machines and VM scale sets. For very large workloads, Event Hubs Dedicated is the option worth evaluating first. Another approach is to use multiple Premium namespaces and shard producers across them in your application, but that adds routing, monitoring, and operational complexity.
That matches my concern. I’m also considering Dedicated, but the hourly price appears to be several times higher than Premium. The relationship between processing units and capacity units isn’t very clear, so I’m not sure how to compare them. If we use multiple Premium namespaces, we could assign specific producers to particular namespaces and similarly route consumers to the relevant partitions. I’m trying to determine whether that architecture is practical before committing to the added complexity.