HACKER Q&A
📣 rishiloyola

What is the best way to calculate percentile of streaming data?


Hello,

I need to code python function which will iterate through incoming requests and calculate percentile of size of body dynamically. Which lib or algo do you guy recommend?

Example: Requests are coming in batches. Let's say first batch has 50 requests, next one has 80 etc. I need to calculate percentile of size of body that each request has.


  👤 sigmaprimus Accepted Answer ✓
I think you need to provide a bit more info, are you using Apache Kafka? Something else?

The function would be individual batch requests divided by total requests multiplied by 100, but I dont think thats what your looking for.

Edit: actually, for your question it would be the inverse of batch size multiplied by 100, eg. First batch has 50 request so that would be 1/50×100 or 2%