I ran a test today that appeared to fail 2% of the time, but according to my assertions which check for less than a 1% failure rate, they were passing:
13:07:32 ================================================================================
13:07:32 2015-05-21 13:07:32 157s elapsed
13:07:32 ---- lightweight GET app pool --------------------------------------------------
13:07:32 [##########################################################################]100%
13:07:32 waiting: 0 / active: 0 / done:500
13:07:32 ---- Requests ------------------------------------------------------------------
13:07:32 > Global (OK=2014360 KO=31620 )
13:07:32 > lightweight GET app (OK=2014360 KO=31620 )
13:07:32 ---- Errors --------------------------------------------------------------------
13:07:32 > status.find.is(200), but actually found 503 31620 (100.0%)
13:07:32 ================================================================================
13:07:32
13:07:37 Simulation finished
13:07:37 Parsing log file(s)…
13:09:09 Parsing log file(s) done
13:09:09 Generating reports…
13:09:09
13:09:09 ================================================================================
13:09:09 ---- Global Information --------------------------------------------------------
13:09:09 > request count 2045980 (OK=2014360 KO=31620 )
13:09:09 > min response time 1 (OK=1 KO=13 )
13:09:09 > max response time 15202 (OK=12794 KO=15202 )
13:09:09 > mean response time 11 (OK=4 KO=436 )
13:09:09 > std deviation 161 (OK=60 KO=1132 )
13:09:09 > response time 50th percentile 2 (OK=2 KO=164 )
13:09:09 > response time 90th percentile 4 (OK=4 KO=717 )
13:09:09 > mean requests/sec 13029.894 (OK=12828.521 KO=201.373)
13:09:09 ---- Response Time Distribution ------------------------------------------------
13:09:09 > t < 800 ms 2011931 ( 98%)
13:09:09 > 800 ms < t < 1200 ms 792 ( 0%)
13:09:09 > t > 1200 ms 1637 ( 0%)
13:09:09 > failed 31620 ( 2%)
13:09:09 ---- Errors --------------------------------------------------------------------
13:09:09 > status.find.is(200), but actually found 503 31620 (100.0%)
13:09:09 ================================================================================
13:09:09
13:09:09 Reports generated in 91s.
13:09:09 Please open the following file: /home/tre/jenkins/workspace/Web_Performance_Tests-kappa-benchmark_AppserverBenchmarkSimulation/target/gatling/appserverbenchmarksimulation-1432238695178/index.html
13:09:09 lightweight GET app: mean requests per second is greater than 2000 : true
13:09:09 lightweight GET app: percentage of failed requests is less than 1 : true
13:09:09 lightweight GET app: 95th percentile of response time is less than 250 : true
I am running version 2.2.0-M2
I don’t have the full simulation log, but I can copy out whatever information you might need to help with this issue.