gatling 2.0.3: debug logging significantly affects response times

Gatling (Open-Source)
1

Greetings,

With gatling 2.0.3 and debug logging enabled [logback.groovy:logger(“io.gatling.http”, DEBUG) ], reported response times for our simulations are close enough to wireshark times to be acceptable for trend analysis (e.g. 15-20ms mean across 12 service calls per user).

If, however, we disable debug logging, the reported response times go up significantly, from 3x to 10x actual, making the response time reports highly suspect.

I have been running various permutations of our simulations (going back in time, etc) to try and isolate the issue. The one common factor is leaving logging enabled ensures valid close to valid response times.

I’ve examined the source code and can find no clear indication as to why the response times would become so inflated when debug logging is disabled.

Our load generators are heavy-duty 32 core, 32GB, server-grade systems in the data center and c3.large in AWS. Load tests have ranged from 1K CCU to 800K CCU.

Any advice or thoughts about how we might normalize the reported response times when logging is enabled or disabled would be greatly appreciated.

Thank you,

–sky

2

Hi Sky,

This is very weird and I’ve never experienced such a thing.

Can you share a reproducer? Have you tried to upgrade to 2.1.4?

3

Hi Stéphane,

Indeed, neither had we! :slight_smile:

After much gnashing of teeth and deep review of changes, we found what we suspect is the root cause: TRACE level was enabled, along with both file & console logging. We manage our load tests from TeamCity, using Fabric to control the remote agents. Our suspicion is that the increased logging on multiple streams was, somehow, starving threads, thus affecting the capture of more accurate timings.

Under very small loads, no starvation effects were occurring, and the results were on par with past simulations. We’ve since disabled virtually all logging and console output, and results are now being reported correctly.

Thanks!

–sky

4

It makes sense, glad you could sort it out, you scared me a bit… :wink: