[gatling sbt plugin] how to generate a test failure if for example 95 percentile response time is greater than a given limit ?

Gatling (Open-Source)
1

Hello,

I’m using gatling-sbt-plugin to build my load test scenario, and jenkins gatling-plugin, to trigger automatically the load tests, and I wonder how I can make the load tests as executed by gatling-sbt-plugin fail when some conditions are not met, for example when the overall 95 percentile response time is greater than a specified limit ?

If nothing already exists, the solution I found would be to get, from the gatling generated report, the file …/js/global_stats.json which contains everything required :

{
“name”: “Global Information”,
“numberOfRequests”: {
“total”: 1484969,
“ok”: 1484969,
“ko”: 0
},
“minResponseTime”: {
“total”: 0,
“ok”: 0,
“ko”: -1
},
“maxResponseTime”: {
“total”: 359,
“ok”: 359,
“ko”: -1
},
“meanResponseTime”: {
“total”: 82,
“ok”: 82,
“ko”: -1
},
“standardDeviation”: {
“total”: 41,
“ok”: 41,
“ko”: -1
},
“percentiles1”: {
“total”: 90,
“ok”: 90,
“ko”: -1
},
“percentiles2”: {
“total”: 105,
“ok”: 105,
“ko”: -1
},
“percentiles3”: {
“total”: 137,
“ok”: 137,
“ko”: -1
},
“percentiles4”: {
“total”: 186,
“ok”: 186,
“ko”: -1
},
“group1”: {
“name”: “t < 800 ms”,
“count”: 1484969,
“percentage”: 100
},
“group2”: {
“name”: “800 ms < t < 1200 ms”,
“count”: 0,
“percentage”: 0
},
“group3”: {
“name”: “t > 1200 ms”,
“count”: 0,
“percentage”: 0
},
“group4”: {
“name”: “failed”,
“count”: 0,
“percentage”: 0
},
“meanNumberOfRequestsPerSecond”: {
“total”: 4115.528198192455,
“ok”: 4115.528198192455,
“ko”: -1.0
}
}

is the file contents stable ? for example the name “percentiles1” is not very meaningfull.

the tested web app : https://github.com/dacr/primes-scalatra-app
the web app container : https://registry.hub.docker.com/u/dacr/dock-primesui/
the load test : https://github.com/dacr/primes-scalatra-app-loadtests

regards,
David.

2

Hi David,

Use Gatling’s Assertions : http://gatling.io/docs/2.1.6/general/assertions.html
If one of your assertions fails, this will be seen as a test failure.

Cheers,

Pierre

3

It will be more a workaround, as it will “hard-code” the assertion into the load test source code. From jenkins point of view it would be better to separate the assertions from the load test code and to be able to set/customize them directly from jenkins user interface (typically for us the assertions depends of the platforms characteristics used for the deployments).

But as a start, I will use the assertions feature with some java system properties for minimalist customization, thanks.

4

Of course, you can always implement your own assertions system by parsing stats.json/global_stats.json.
But if Gatling’s Assertions, only customized through some environments variables/system properties proves sufficient, it might not be worth it to reinvent the wheel :wink:
Note that even if it it not likely that the structure of those files will change, we do not guarantee that they won’t change either.

Cheers,

Pierre