How much distributed tracing costs:Using OSS like Jaeger or paid product

Question

Wanted to get your inputs on 2 solutions for tracing needs: 1. Open Source solutions like Jaeger - Is there a blog or link which analyses the cost of storage of traces to any of the backend like Cassandra or Elastic? Do you use any compression techniques before storing to DB?2.Paid products like Datadog - Most of the vendors charge on a per-host basis. What's the underlying logic for charging like that?If I were to use a vendor for this, I would like to be charged according to the number of spans sent for storage like in logs, say $2 per million spans or $0.2 per GB. Only Datadog seems to charge on such basis and asks for $1.7 per million of spans apart from $31 per host which covers 1M spans only. Does DataDog give you control of which spans to visualise & store?- Does any vendor give you control of spans to process or a clear pricing estimate for the tracing part?What other things should I look when using Jaeger or buying Datadog? My primary need is to monitor and debug my applications.

akyaky · Accepted Answer

Hey! I can't speak to the exact mechanism for pricing on a large or enterprise level account, but it sounds like you're just getting started with tracing - might be worth checking out Lightstep. Pricing is pretty transparent for different tiers of non-enterprise usage and it's seats+services monitored, NOT consumption, so there shouldn't be any end-of-month surprises. (https://lightstep.com/pricing)
Lightstep and several other vendors use open source standards so worst case scenario, you have great instrumentation even if you're not paying for a product right away. /shrug
(full disclaimer, I work at Lightstep albeit not in sales)

verdverm · Answer

The thing with Jaeger is you will likely need to deploy other services with it. For example, if you use Istio, pods need sidecars with non-negligible resource requirements. You'll need EFK / Prometheus.
You should include the salaries to deploy, debug, and maintain in the calculation. Not just Jaeger, all the supporting systems too. Finding people (you'll need more than one most likely) is a challenge as well.
You probably don't need distributed tracing at first, I have only seen a few people at this point. Start with basic logging and metrics if you have none today.

tannerbrockwell · Answer

Look at OpenTracing. This is the merger of two tracing initiatives and will provide the pointers to both commercial and open source solutions. [1] There is some excellent capabilities to instrument an existing java app through the jvm [2] with OpenTracing.[1]: https://opentracing.io/ [2]: https://medium.com/opentracing/opentracing-on-kubernetes-get...

tj_9000 · Answer

Hello, I am a Civil engineering student in my final year of Bachelor's degree. I am very interested in the field of machine learning and ai.But i am trying to determine how can i integrate my academic knowledge with machine learning. Any suggestions or ideas would be greatly appreciated.Thank you.