I'm working on a trading platform where we have a service that processes medium-rate data (4Hz with 1000 items) and delivers it to numerous websocket clients after some filtering. The goal is to provide updates to clients within a few dozen milliseconds of the initial data arrival. Currently, we're pushing the original data into a Kinesis stream and then using a separate service with enhanced fan-out to process and stream this data to the websocket clients. While this setup works for now, we're hitting a limit with the number of websocket clients we can support, as Kinesis enhanced fan-out restricts us to only 20 registrations. Beyond that, latency without enhanced fan-out skyrockets to over 2 seconds, which isn't acceptable for us.
We're considering shifting the Kinesis reading and processing to a third service equipped with a gRPC interface to push updates out. This way, each gRPC server could manage hundreds of connections, allowing for easier horizontal scaling. However, this solution feels like it might be reinventing the wheel when AWS might have existing solutions. I'm looking for alternative methods to achieve our goals without hitting these limits. Any ideas or suggestions?
1 Answer
From my experience in sports betting and trading platforms, a straightforward Golang HTTP server can handle up to 100,000 messages per second with a latency around 300 microseconds. I'd suggest removing all AWS-managed services if reducing latency is your priority; using bare EC2 instances directly without a load balancer is one of the fastest ways to go. Just be cautious as adding any service, like RDS or Kinesis, will introduce delays.
I’d love to see your source code, that sounds really interesting!
For higher performance (millions of transactions per second), consider using bare metal instances and ensuring your instances are close together physically to minimize latency.