Possible discrepancy between generated and computed percentile99 values

Hello all,

This is my first time to post here in this group and hope you guys are well.

I found a discrepancy (diff of 51ms) between the generated percentile99 value in the gatling generated static html and in grafana (given received metric values and shown as percentile99). Maybe due to computation/conversion with round-offs, but I need to verify, since 51ms in multi-user sampling will be hundreds or thousands of ms diff when going to more user sampling.

link A: https://www.emathhelp.net/calculators/probability-statistics/percentile-calculator/?i=.727%2C1.559%2C1.823%2C1.967%2C2.095%2C3.055%2C3.551&p=99&steps=on
link B: http://www.statisticshowto.com/quantile-definition-find-easy-steps#text-14

  • Either formula is supposed to return 3551ms even if it uses math.round() function.

My questions are:

  • Is 3551ms the better value to present than the ones showed in static html summary report?
  • Above are all on how I understand the flow, please do correct me if I’m wrong. Will that 51ms difference is due to conversion from sec to ms and back from ms to sec for presentation?
  • I’m still not convinced regarding the math.round() function as I believe it has to declare some format at what point decimal it should start rounding off, would somebody point me the specific code block that does the rounding off which leads to slight discrepancy?

Gatling highcharts v=2.3.0
Gatling maven plugin v=2.2.3

Any reply would be much appreciated!

Thank you much,
Mike

There are many differences between HTML reports and the Graphite integration:

  • the former uses t-digest while the latter uses HdrHistogram
  • the former split the total time window in 1000 buckets by default, with min 1s, while the latter uses constant 1s buckets, so grouping can be different
  • InfluxDB is very ill suited for distributions as it can only store numbers, hence percentiles, so you’ll end up doing plain wrong maths as soon as you need to aggregate values (eg averages of percentiles don’t make any sense).
    In any case, you’ll get approximated values. Computing exact values would require retain all values in memory and sort them, which is simply not feasible as Gatling needs to support use cases with 100,000 rps.

That being said, FrontLine comes with great real time distributions computation without the Graphite/influxDB flaw.

Hello Stephane,

Thank you very much for the explanation, this enlightens me and thank you for suggesting Frontline.

Regards,
Mike