What are the hard problems I can expect if I try to build realtime features into our product?
Should I even try to build it or look at third party suppliers? I've seen various articles that compare DIY with buying it in, but they're from those suppliers and I wonder how neutral they are.
It's obviously complex, but what are the primary engineering challenges? Message delivery ordering/duplication and missing messages? Is it security/authentication or problems with debugging and observability? Or should I worry most about infrastructure, scaling and performance?
If you just get some amazing idea you want to quickly lauch and see the result, then the best way is to adpot 3rd parties as much as possible (as long as those stuff could work together).
Then your product receives the momentum, exciting. But you start to have all kinds of issues because you don't spend much effort on the stuff, the initial version. It's not about your capability. You just don't need a perfect design in the very early stage of the product. You might only need 1 or 2 people in order to get all the SaaS work together, although it's not optimal from system and computer resources perspectives.
Then you start to think about refactor the product, a better design, a decent system which means you have to worry about everything. And the first problem is do you have adequate engineers to do that work.
Some startups died in the refactor...
For realtime features, it's hard to build since you have to worry all the infrastructure, scaling and performance. And the even harder part is to run the realtime features smoothly and stably. So I would recommend to use 3rd parties SaaS until you have enough resources and time.
Of course, reddit is a huge company and has loads of established engineers. YMMV.
Or do you mean an event driven web based system? The term realtime has become overloaded and your answer changes the challenges… a lot.