I'm currently developing a high-speed distributed database using Rust, and I'm utilizing technologies like io_uring, eBPF, and the NVMe API. Unfortunately, this means I can't rely on most libraries or frameworks, so I'm building everything from scratch, including a custom event loop. I've managed to implement Unix Domain Sockets (UDS), UDP, and TCP without TLS/SSL due to my limited experience with those, but I want to keep the question general enough to cover different protocols.
Here's my scenario: Alice connects to my database and sends two commands simultaneously: `SET KEY1 PAYLOAD1` and `SET KEY2 PAYLOAD2`. Given that these payloads can be quite large, they may exceed the packet size limit. How can I efficiently handle this situation and identify that the two packets belong to the same command? One idea I had was to include a RequestID or SessionID in each packet, but I'd need to know where a message might split. I'm concerned about detecting the MTU, as it could be inefficient. What strategies could I employ to manage this?
1 Answer
If you're having trouble with what seems like a common UDP issue, just remember that building an ACID-compliant database is going to be significantly tougher! But I get that everyone starts somewhere, so keep pushing through!
Thanks for the reality check! I appreciate the encouragement, even if I disagree. We'll see how this goes.