I'm looking for ways to enhance the redundancy of important data transmitted over TCP/IP. When using TCP/IP, the sender gets an acknowledgment back for each package, but I'm concerned about potential issues like power loss or voltage spikes after an acknowledgment is sent. AWS has mentioned that data should be distributed across three Availability Zones (AZs) for security, so I thought about implementing a route-splitter that would send packages to three different targets, waiting for all of them to acknowledge before reporting back to the sender.
Similarly, for sending packages, I'd like the splitter to forward them to two backup locations and only proceed to the main target once they've acknowledged receipt. This approach is particularly relevant for communication across different companies or over large distances. I've worked in environments where companies had preferred data centers, but their operations were spread out over vast areas, and disruptions like fiber cable attacks are becoming an increasing concern. One of our partners faced significant cloud egress costs, complicating matters further since HTTP GET requests were no longer idempotent. It feels like this situation mirrors the careful data handling done in scientific experiments or customer interactions, where capturing every piece of raw data is crucial for analysis later on.
4 Answers
In essence, you want something similar to how private torrent clients work, right? Although keep in mind that your partner's specifications may limit your options.
Your question is pretty complex! There are plenty of storage and database solutions that handle replication and guarantees, but it's essential to clarify your goals. Are you looking for high availability or just curious about redundancy techniques? TCP itself can't handle the acknowledgments in the way you want under your scenario, but something like a message bus could help with delivery assurances without diving too deep into the TCP layer.
Exactly! Message buses often provide the kind of delivery guarantees you might need without worrying about the lower-level TCP mechanics. But make sure you define your project's requirements clearly.
What you're looking for is basically a distributed fault-tolerant delivery system. The good news is that there are existing solutions for this. However, it’s important to understand what technology is already out there and how it can fit into your specifications. It sounds like you’d need a kind of message bus or similar technology to achieve your redundancy goals.
That's interesting! But do you know what this kind of system is typically called? I can't use technologies like MQTT since our partner has set specific requirements.
It's worth noting that TCP doesn’t support the kind of redundancy you’re after directly. It manages acknowledgments in the TCP/IP stack, so you would need to look at higher-layer protocols for this type of functionality.
Yeah, but unfortunately, torrent protocols are not on the table with our partner’s setup.