Gatling reports shows a request with response that never happened to server

Gatling (Open-Source)
1

Hello There,

Need a help from you. I saw something strange in my Gatling test, where the Gatling report is showing a one request took max time of 20 secs response but when I try to see the same call/request in the logging or monitoring tool, that call never happened to the server. In fact, I observed this happened for couple of requests.

I verified thoroughly(including graphs) during the test time frame but no trace found at all.

Just wondering what could be reason.

Thank you :blush:

2

What’s the status of this request? Are you sure it’s not a connection or TLS handshake timeout? If that’s the case, it’s obviously not visible in the server logs as it never made it there through the network.

3

@slandelle The request is successfully executed with response code 200.

We are actually using Datadog for app server and AppD for DB monitoring.
Is it possible that request took 100% of execution time from DB hence I’m not seeing this in app server logs?

4

I think that you have cached this request:

By default, Gatling caches responses based on:

  • Expires header
  • Cache-Control header
  • Last-Modified header
  • ETag

You can disable this feature.

5

@GeMi - I am sorry I didn’t understand much from it. Can you please elaborate!

And, I already have “flusHttpCache” and “flushSessionCookies” in scripts. Please let me if I need to add anything else which helps to avoid the issue.

Thanks

6

Do you have:

http.disableCaching();

in your HttpProtocolBuilder instance ?

7

This sentence is very confusing:

Gatling report is showing a one request took max time of 20 secs response but when I try to see the same call/request in the logging or monitoring tool, that call never happened to the server.

@Vhrlrng Do you mean that:

  1. you haven’t found a request with a 20s response time in your server logs/monitoring?
  2. you identified the specific request execution (timestamp, parameters) that took 20s and couldn’t find it in your server logs/monitoring?

I suspect it’s 1.
In this case, you have to understand that the client side response time is not equal to the server side one:

  • lag in your load generator machine (ex: running on Windows with antivirus on :man_facepalming:, saturated CPU, not enough cores )
  • DNS
  • TCP connection
  • TLS handshake
  • network hiccup
  • load balancers/reverse proxies

You won’t find any of those in your server side monitoring.

8

No @GeMi. will add that as needed.

So, 2 or more requests were added up and showed as the single request!

9

Correct. @slandelle

True and I am trying to find more possible reason. :slight_smile:

Thank you.

10

Correct. @slandelle

My sentence was a XOR: 1 XOR 2, not both.

you identified the specific request execution (timestamp, parameters) that took 20s and couldn’t find it in your server logs/monitoring?

So you’re saying that Gatling logged a request that received a response with a 200 that you could not find in the server logs nor monitoring?! The only way this is possible is that the response was served by a cache in front of what you’re monitoring, or that your monitoring is buggy/misses some requests.

Sorry, but that’s not something we can help with, all the more without you providing a way for us to reproduce on our side. But I’m 100% convinced the issue is not on Gatling’s side.

11

Hi @GeMi - I did implemented this in my simulation and see still issue is persist.

image
image691×203 44.1 KB

would you mind take a look at it!

Thanks

12

@Vhrlrng I understand that @GeMi 's suggestion was more convenient for you so you wanted to give it a try instead of mine, but it’s wrong.

It would explain requests being skipped (because cached), not long requests not visible in your server logs.

13

@slandelle - I am trying all the ways to understand the issue as I have very less time either to resolve or explain my team that why it occurred.

Also, working with APM team to verify if monitoring tool is not capturing everything as it shows in Gatling,.

Thanks

14

I think you have here 2 additional options:

  • Run tests by any proxy and check if all requests go by it,
  • Install Wireshark or any tcpdump tool to check trafic durning test.