I have a client connecting to a server. Currently communication is via raw TCP / JSON messages. There are known reliability issues, and somewhat hacky suggestions of how to fix this.
Instead, I know open-source messaging frameworks exist, like ZeroMQ, RabbitMQ, Apache Kafka, probably others. They all advertise features like low-latency communication, routing, complex messaging topologies etc. These are not my primary concerns - I only have one server and one client. What I care about is
- Reliable delivery - messages are not silently dropped, and delivered in correct order
- Tolerance to crashes of either client or server (e.g. messages not consumed by the client get seamlessly replayed); e.g. the framework keeps track of messages written and read by client
- So long as latency, throughput, memory footprint etc. are not extremely unfavourable, these are non-objectives; typical required message throughput is less than 1 msg/sec, with maybe bursts of 10-20 of messages/sec for a few seconds.
- Ease of integration into a C++ (server) and Python (client) code base, and general ease of maintenance - if the framework is a very complex beast with a steep learning curve, it's not necessarily better than hand-crafting the TCP protocol.
Basically, I want a queue, into which server can push, client can pop from, and reliability is guaranteed by the framework. Does any OSS framework help my case? I tried googling around, but mostly get SEO garbage articles, or ones that focus on features I don't need. I also tried SE, but apparently asking about appropriate software choices is off-topic - go figure...
But, after read your description, I could not recommend it to you, could be an operational burden...
The options that come to my mind (without entering into details): Redis pub/sub or PostgreSQL