Different results between two same gatling tests

Gatling (Open-Source)
1

Hi,

I have a small problem and I would like to know the cause. I did a T1 test a few days ago on my project. The results were consistent, I was happy. The weekend is over, I decide to redo this same test (which we will call T2) (identical script and parameters). Problem: the results are completely different (my response times are much better and I have no errors, contrary to T1) :confused:

I thought that this difference came from the tests I could have done before T1: my server would not have had time to recover from the previous tests and the returned results would be inconsistent. But I tried again and the difference is still there between T1 and T2 :confused:

I also submitted the hypothesis that, during T1, several background tasks were taking up bandwidth and could have influenced the test… But again, I tried to run T2 again but this time with connection-intensive sources in the background, but nothing happened.

Could you tell me the cause of this difference?

Thank you !

AG

2

Hi @ArthurG,

Sorry, but I won’t be able to help you on this one. But let’s try to find a possible cause together.

If you have different results between T1 and T2, it means something changed during the meantime.

I know that a lot of organizations take advantage of weekend to upgrade either their OS, dependencies, softwares or hardware. It may impact your result in a way or another.

The bandwidth reason is perhaps in your whole organization, not only from your computer.

Maybe your server were full and during the weekend a restart clean their (erroneous?) cache.

As you had longer response times and some errors on T1, perhaps it was during a db backup?

Again, I’m sorry, but really, without knowing your context, it’s hard to tell you what is the root cause.

I hope these questions help you to find it yourself.

Cheers!

3

Okay, I see !

Thank you for your answer :slight_smile:
AG

4

Did you find something useful?

5

No but I will try with another solution: execute my tests in a VM !

6

You want to run your tests in isolation:

  • your load test generator mustn’t compete for resources (CPU, memory, bandwidth) with other processes on the same machine (eg run at the same time multiple load tests or Selenium/cypress or compiling code :man_facepalming: )
  • your target application mustn’t be performing other heavy task at the same time, unless it’s a desired test case (eg load tests during nightly batch operations)