Maven Gatling and Assertions

Gatling (Open-Source)
1

Hi,

I’ve just started using Gatling, and I have created 2 simulations.

I am using the maven Gatling plugin to execute the tests.

All the simulations run sequentially which is fine, the problem lies in that if simulation 1 assertions fail the build fails and it no longer runs the 2nd simulation.

If only a check fails then the 2nd simulation will run as expected.

Is there anyway to make sure that all simulations are always run regardless of whether the first simulation fails it’s assertions?

I cannot understand why assertion failures cause the build to fail.

Thanks

Ron

2

Looking at the source for the maven plugin, it doesn’t look like there’s a way around this. The simulations are run in a loop here: https://github.com/gatling/gatling-maven/blob/master/gatling-maven-plugin/src/main/java/io/gatling/mojo/GatlingMojo.java#L194. If any assertion fails, the method “executeGatling” throws an exception and we leave the loop. I think it would make sense to change this to allow for continuing to loop through the simulations even if an assertion fails, I guess depending on some config value (though I’m new to gatling so I may well be thinking about this all wrong!). I’d be happy to work on such a change myself but I’m not sure where the best place is to propose/discuss changes - if anyone could point me in the right direction (or let me know if the answer is “right here”) that would be extremely helpful!

3

Hi,

Look a little bit closer : https://github.com/gatling/gatling-maven/blob/master/gatling-maven-plugin/src/main/java/io/gatling/mojo/GatlingMojo.java#L104-L111 :slight_smile:
This behaviour can be disabled by setting false in the plugin configuration.

4

Unfortunately after trying this it doesn’t work, as the exception is still thrown and failOnError is not evaluated till we leave the for loop which iterators around all the simulations.

catch (Exception e) {
if (failOnError) {
throw new MojoExecutionException(“Gatling failed.”, e);
} else {
getLog().warn(“There was some errors while running your simulation, but failOnError set to false won’t fail your build.”);
}

So now the build says success but it still doesn’t run the subsequent Scenarios

5

Hi Pierre,

Thanks for your reply :). I think the behaviour that Ron described above is different to the failOnError behaviour. I’ll describe what we’re talking about in more detail, to make sure I’ve explained properly:

Say we have two simulation classes in our project, FirstSimulation and SecondSimulation. In our pom.xml, we’ve set true in the plugin configuration. When all the assertions are passing, our maven build runs FirstSimulation, then SecondSimulation. But if an assertion fails in FirstSimulation, then SecondSimulation doesn’t run. We’ve tried this with failOnError set to false, and although the build no longer fails, SecondSimulation still doesn’t run.

Looking at the source code, I think what happens is:

  1. We enter the for loop over all simulations, at line 193 (the line above above my previous link)

  2. FirstSimulation is executed, and an assertion fails

  3. A GatlingSimulationAssertionsFailedException is thrown

  4. We go straight to the catch block on line 197, i.e. we leave the for loop. This means that we’ll never execute SecondSimulation

  5. Now we check if failOnError is true, and decide whether to fail the whole build

I hope that’s clear! Our desired behaviour is that SecondSimulation always runs, even if an assertion fails in FirstSimulation. Do you think it would be sensible to allow that behaviour (perhaps controlled by a new config value)? I’d be keen to contribute code for a change, if you agree that it’s a good idea :slight_smile:

Ed

6

Bump

7

Contribs welcome :slight_smile:

8

Hi Ronald,
I would be interested to know if there was any solution was conculded for the below issue.
As I am facing similar issue in my project where i am executing more than 20 simulation API Test and my build fails everytime if anyone of the API Check fails and stops from the execution of remaining simulation files.
with the similar error.

Failed to execute goal io.gatling:gatling-maven-plugin:2.1.7:execute (getPrincipaltradenames) on project BOND-microservice-gatling-tests: Gatling failed. Gatling simulation assertions failed ! Process exited with an error: 2 (Exit value: 2) → [Help 1]

so if there is a way where i can conitnue my tset to run.
you response will be much Appreciated.

with regards
Afshan.s

9

Hi,

I had troubles finding this information, so I post it here.

As of v2.2.2 of the Gatling maven plugin, there is a new configuration key continueOnAssertionFailure which does exactly that.

See the commit, here.

10

Do we have similar configuration for gatling gradle plugin?

11

We don’t. But a contribution would be appreciated :slight_smile: