realtime/graphite integration - documentations vs reality

Hello

I integrated my Gatling tests with Grafana + InfluxDB instance. Overall it works great - the metrics are sent and I can watch at real time the progress of my load tests. But I noticed some discrepancy (I think) between what is in the generated report/documentation/grafana.

In this particular scenario I am ramping up to 600 users per second during 60 seconds and then keeping this load also for 60 seconds.

In the docs it says that:

Active Users

At a given second, active users are computed as:

  • the number of active users at the previous second
    • the number of users who started during this second
    • the number of users who finished during the previous second

And by looking at the Gatling HTML report active users (orange line) it looks ok - around ~600 users and at the end number of active users increases because my machine can’t keep up with the requests. On the other hand on my grafana dashboard I can see that most of the time the number of active users is 0, only when my users are starting to stack up I am getting the number of active users != 0.

The second thing. In the docs it says that

Request metrics

Requests metrics give you, for each request (or all requests using allRequests) and by status (OK, KO and both) :

  • count : requests/sec
  • min and max : Min and Max response time for the request
  • percentilesXX : 1st, 2nd, 3rd and 4th percentiles, as defined in gatling.conf (defaults are 95th and 99th percentiles)

but comparing grafana and generated report I can see that the “request-metrics” graph is more similar to the “Number of responses per second” one - and looking at the code of GraphiteDataWriter it records the response not the request event:

override def onMessage(message: LoadEventMessage, data: GraphiteData): Unit = message match {
case user: UserMessage => onUserMessage(user, data)
case response: ResponseMessage => onResponseMessage(response, data)
case _ =>
}

As I understand the first one is working as intended: https://github.com/gatling/gatling/issues/2178. Users are not reported as active if they die between the start and end of the send interval, but the seconds case (I think) may need code/documentation change if my assumptions are correct.

Hi

On the other hand on my grafana dashboard I can see that most of the time
the number of active users is 0, only when my users are starting to stack
up I am getting the number of active users != 0.

Can you reproduce with current master/snapshot?

but comparing grafana and generated report I can see that the
"request-metrics" graph is more similar to the "Number of responses per
second" one

You're absolutely right. I fixed the doc:
https://github.com/gatling/gatling/issues/2746

Thanks.

*Stéphane Landelle*
*Lead developer*
slandelle@gatling.io

I will try to reproduce the problem today or tomorrow and post the results here.

W dniu poniedziałek, 8 czerwca 2015 20:07:57 UTC+2 użytkownik Stéphane Landelle napisał:

Great, thanks!

Better late then ever.

I wanted to use newer version of Gatling to test the active users that are pushed to graphite. The thing is - I have normal Scala project with gatling-sbt-plugin which I am using to execute gatling as a test framework. I can see that there are snapshots of Gatling: https://oss.sonatype.org/content/repositories/snapshots/io/gatling/highcharts/gatling-charts-highcharts/ but I can’t find “snapshots” of gatling-sbt-plugin. Will 2.1.6 version of gatling-sbt will work with Gatling 2.2 after changing versions of

"io.gatling.highcharts" % "gatling-charts-highcharts" % "2.1.5" % "test"
"io.gatling"            % "gatling-test-framework"    % "2.1.5" % "test"

?
W dniu poniedziałek, 8 czerwca 2015 21:35:56 UTC+2 użytkownik Stéphane Landelle napisał:

but I can't find "snapshots" of gatling-sbt-plugin.

AFAIK, sbt plugins have to be deployed in an Ivy repository and can't be
deployed in a maven one. And I'm not sure Bintray provides a repo for
snapshots. Sadly...

Will 2.1.6 version of gatling-sbt will work with Gatling 2.2 after
changing versions of

"io.gatling.highcharts" % "gatling-charts-highcharts" % "2.1.5" % "test""io.gatling" % "gatling-test-framework" % "2.1.5" % "test"

Yes.

I am having the same problem with 2.2.0-snapshot.

Here is the image from report: http://i.imgur.com/dn5b4Fs.png and from grafana: http://i.imgur.com/VTlm0mn.png. It seems to me that in report we have something like: https://github.com/gatling/gatling/issues/2305 because this loadtest is very simple: inject(constantUsersPerSec(maxUsers) during(time.seconds)) but in the grafana I see the users that are reported inside the box in the console: something like:

users who were alive during the previous second
- users who died during the previous second

Could you try the fix I pushed, please? https://github.com/gatling/gatling/issues/2750
It should be up on Sonatype in ~20 min.

I’ll wait for your feedback before backporting on 2.1.

Thanks,

I checked the latest SNAPSHOT and it works - the active users metrics reported inside HTML report and onto graphite are the same, thanks!

Thanks for your feedback. I’ll backport on 2.1 then.

Cheers,

I found your github repo and like what you have done for AWS live metrics. I have a question. Were you able to see performance metrics for all the load generators?

Thanks,
Abhi