runMultipleSimulations: subsequent simulations not run when assertion failure occurs

Gatling (Open-Source)
1

Hello!

When using (gatling-maven-plugin 3.0.4), I’ve found recently that an assertion failure in any of the simulations will cause subsequent simulations to not execute.

To illustrate, I have three independent simulations: Sim01, Sim02, and Sim03. Each has a global assertion for response time <2000ms on a unique HTTP GET. If Sim01 fails its assertion, Sim02 and Sim03 will not run.

I’m using mvn gatling:test from a Windows terminal to execute the simulations. I’ve validated by commenting out the failing assertions and re-running. When there are no assertions, or assertions succeed, all simulations execute.

I experimented with the plugin and configured includes to “whitelist” the simulations I wanted run using both explicit naming (e.g. simulations.Sim01 simulations.Sim02, et cetera) and implicit naming (e.g. (e.g. simulations.*). The behavior was the same. When an assertion failed in a simulation, no other simulations would execute.

Is this behavior expected? Should a failure in a preceding simulation halt the execution of remaining simulations when runMultipleSimulations is set to true?

Thanks for your time.

-Carl

2

runMultipleSimulations was introduced as a hack as some people didn’t want to configure multiple CI jobs.
Behavior is expected: assertion failure is a blocking failure.
I recommend you run multiple CI jobs, one for each simulation.