We are currently using Gatling to create some background user load, to be kept running while other tests are being executed, and the scenarios that are defined in a way that the users will have some randomized branching within them. For this we’re using uniformRandomSwitch at some steps to do calls to different endpoints. Since the requests are being randomly selected, sometimes we are obviously getting Could not find stats matching assertion path List(X, Y) in our HTML reports and JUnit results.
Is there a possibility to not even creating the assertion for them if the request was not even executed? I think the list of assertions are already being generated at the start so they can be used during the run when they are hit, so there might not be a way of completely not showing them.
The other idea would be to use the roundRobinSwitch, which would solve this problem with longer runs, ensuring that all the requests under the switch would be executed. However, with shorter runs, for example PR merges that runs each scenario once, the issue can still be observed because the other requests will not be executed. Additionally, since it is a round robin action we will always call the first request under the switch, with shorter runs.
I’m not sure to understand your question. Can you provide a small sample of code?
What I understood is that you have a bunch of possible path but not all are available for a user. Can you write a test for each path if the path is possible or not before calling it?
That way, you may still use uniformRandomSwitch but when the random choose an unavailable path, you don’t execute it and directly loop on another branch.
First, the assertions are computed when the Simulation is loaded while in your case the requests actually executed are known at runtime.
Then, we don’t currently have an option for marking an assertion as “this request was never executed but this is expected, just ignore it” instead of behaving like “this request was never executed, this is indeed a test issue”. That’s an option we could add in a future revamping of the Assertion API. This will be a breaking change so that’s something we want to do all at once.
Finally, uniformRandomSwitch to visit all the paths under load so I guess you only experience this issue when running a smoke test, am I right? In this case, you could probably only add the assertions when running a full scale load test with a System property.
@sbrevet
Indeed expanding the scenarios with additional flows to include static paths for each random branching out could be a solution, although could result in some duplicate code, since the scenarios are carrying many session variables depending on the previous actions. However, we will take a look into it and probably need to do it anyway, regardless of how pretty they will end up.
@slandelle
I was mainly thinking whether we could somehow omit the assertion with either a gatling.conf setting that I am not seeing, or maybe a hidden property like .optional() to get rid of them, although couldn’t find it in the base gatling classes.
First, seems like exactly as I thought, that they are created on startup.
Then, I will keep my eye on the release notes, if it ever comes up, thanks for the detailed information.
Finally, Indeed yes they are coming up with a sort of smoke testing, we tried to cut some corners and mostly just used the same scenario flows but only using the build parameters to change our targeted ScenarioBuilder and PopulationBuilders. I guess the correct approach would be to properly categorize our tests via our .pom instead.
Appreciate the technical clarification and the discussion.