question over sharing data between scenarios / before clause

Gatling (Open-Source)
1

Hello Stephane (or whoever can give some insight, perhaps philosophical, on this),
while searching for a clean way to pass data from a before clause or between separate scenarios, i ran across one thread called “share connection between several scenarios”, among others.

Sounds like there is no straightforward way to pass data between scenarios in Gatling.
In fact, in that thread, one of Stephane’s answer is:

We don’t support this use case out of the box, (IMHO, more of a functional test than a stress test).

My question is “why not?”, why is it functional and not stress… could you expand or elaborate on that answer…?
I am curious about it, because now i am questioning our use of Gatling with our microservices, maybe using it outside of its pertinent use cases.

On one end i can see why this is outside of Gatling responsibilities, but in practice, most of the scenarios are going to entail some login or some tokens that need to be provided before the test at hand.

I guess in the end what i am asking is: are we using the wrong tool for the job of load-testing a specific microservice or service call that needs some “before{}” set up?
Is Gatling designed only for mimicking entire user scenarios (all the way, say, from the login) and not so much for “unit-like” performance testing of smaller portions of functionality?
When doing performance testing, should we always test the system in its entirety (be it an open or closed workload system) and not sub-portions of it, perhaps?

Or maybe am i missing something, because i was not able to pass data to the scenario from the before clause… but if that’s the case, what is the purpose or the typical use of a before clause?

Max

2

Hi Max,

There are some misunderstanding here.

Regarding the before and after hooks.

Those are intended for letting users run any arbitrary code before the Simulation starts and after it ends.
Do whatever you want in here: download from S3 some very large feeder file that you don’t want to store along your test sources, perform an initial authentication that you’ll reuse for all your users because you can’t hammer your authentication layer, etc.

What is not possible in here is to use Gatling DSL. If you read the documentation, you’ll see that those are builders and don’t don’t anything by themselves. They are used by the setUp method.

Regarding sharing data between virtual users

It’s of course possible, you can have some virtual users store some data in a ConcurrentHashMap and some others grab data from there. But those use cases are complicated and you’ll have to deal with he orchestration.

Regarding sharing data between Simulations

So now, here, I indeed believe this is a very bad practice.
First, tests should be standalone and don’t depend on other tests.
Then, the reason people usually try to do that is to run a simulation that would clean up the data created by the first one.
We’ve been having databases restore mechanism for ages for this, and now we are in 2018 and can easily spawn trashable platforms (containers, VMs, whatever). This way of performing clean up is all the more very bad as performing mass inserts then mass deletes doesn’t revert a database in its initial state (stats will be off, databases such as Cassandra will be full of tombstones, etc.).

Hope it helps

3

Hi Max,

There are some misunderstanding here.

Regarding the before and after hooks.

Those are intended for letting users run any arbitrary code before the Simulation starts and after it ends.
Do whatever you want in here: download from S3 some very large feeder file that you don’t want to store along your test sources, perform an initial authentication that you’ll reuse for all your users because you can’t hammer your authentication layer, etc.

What is not possible in here is to use Gatling DSL. If you read the documentation, you’ll see that those are builders and don’t don’t anything by themselves. They are used by the setUp method.

yes, that was clear to me from the documentation, but maybe i am missing something else, which may be obvious… let me ask…
If i save a csv file during before{}, it is not clear to me how that can be picked up as a feeder by the scenario.
Example, snippet of Simulation class:

before {dumpFile(target/file.csv)}

val feeder = csv(“target/file.csv”)

val scn = scenario(“MySimulation”).feed(feeder).exec(http(…)

setUp(scn.inject(atOnceUsers(1))).protocols(httpConfig)

That code will throw IllegalArgumentException: Could not locate feeder file, which is correct, because feeder has to be evaluated and csv will read the file, which is not there yet. I need csv() to read the file after before{} completes.

Regarding sharing data between virtual users

It’s of course possible, you can have some virtual users store some data in a ConcurrentHashMap and some others grab data from there. But those use cases are complicated and you’ll have to deal with he orchestration.

exactly, i was trying to avoid that.

Regarding sharing data between Simulations

So now, here, I indeed believe this is a very bad practice.

yes, I’d like to remove that code.

First, tests should be standalone and don’t depend on other tests.
Then, the reason people usually try to do that is to run a simulation that would clean up the data created by the first one.
We’ve been having databases restore mechanism for ages for this, and now we are in 2018 and can easily spawn trashable platforms (containers, VMs, whatever). This way of performing clean up is all the more very bad as performing mass inserts then mass deletes doesn’t revert a database in its initial state (stats will be off, databases such as Cassandra will be full of tombstones, etc.).

yes, then all i need is a before{} that works for me. Currently, i am probably missing something (perhaps obvious… some way to lazy load…? i tried use def instead of val…) on how to use before{} correctly in order to save a file first and then use it as feeder.
Thanks for your answers here!

4

Was able to make before{} run before the feeder by creating the following custom feeder:

val feeder = () => {
val g = envOrElse(“GATLING_HOME”, “target”)
val dir = s"$g/user-files/resources"
val filename = s"$dir/myfile.csv"
Resource.resource(filename) match {
case Success(resource) => (new SeparatedValuesFeederSource(resource, CommaSeparator, DefaultQuoteChar)).feeder(FeederOptionsString, configuration)
case Failure(message) => throw new IllegalArgumentException(s"Could not locate feeder file: $message")
}
}

Unsure how to do any better. Only drawback is that now, by copycatting code from FeederSupport.csv, i can no longer use “circular” or “shuffle” - i have to recreate/copy that logic.

This seems something should be all in the library, not custom code: please advise, and i can possibly go ahead and create a patch.
The current documentation (https://gatling.io/docs/3.0/session/feeder , csv) seems to suggest to use a custom feeder.

5

… I guess this all gets down to one question… let me make sure i did not oversee anything, because it does seem like a very typical case that is not covered by the API.
(in fact, also Stephane states above the typical case of before{}: “download from S3 some very large feeder file that you don’t want to store along your test sources, perform an initial authentication that you’ll reuse for all your users because you can’t hammer your authentication layer, etc.”):

how come there is no feeder that can read, say, csv data, after it has been saved in a new file during a before{} clause?

Asking in case i missed something from the API’s (DSL) use.
Max

6

The actual execution order is:

  1. create the Simulation instance
  2. call what was passed to the before method
  3. execute the test from what was passed to the setUp
  4. call what was passed to the after method

Then, you have to understand that everything that you write directly in the Simulation class body is actually in the constructor, hence executed in step 1.

The csv implementation is pretty complex and it would be too hard for us to try to delay it to step 3.

For your use case, you have 2 possibilities:

  1. create an empty file in the Simulation’s body so the checks pass and then replace it in “before”
  2. directly create your final file in the Simulation’s body
7

Thanks, Stéphane, for your help. Yes, the execution order was clear to me, but i found that the issue had to do with the fact that “.circular” was a macro, in the end.

The feeder csv() was not the issue, like i had mistakenly shown in my code snippet above. In my code snippet I should have copy/pasted the feeder as csv(“target/file.csv”).circular, not as csv(“target/file.csv”).

Going to explain in case other people ran into the same problem, re-summarizing it all here for convenience.

The original issue was that in my code i had one Simulation (Simulation1) that was run with the only purpose of creating a file to be fed to another Simulation (Simulation2), run subsequently. This is bad practice as it creates dependencies between Simulation runs, so i wanted to remove that.
At that point, Simulation2 had this in its scenario:

val scn = scenario(“sim2”)
.feed(csv(“target/file.csv”).circular)
.exec(…)

where file target/file.csv is supposed to be saved by a previous run of Simulation1. I then tried to place code in the before{} clause of Simulation2, using asynchttpclient mimicking the logic of Simulation1 to gather whatever necessary feeder data and dump it into the same file under the target directory. Same logic, but run all within Simulation2: no need for Simulation1 anymore, problem would be solved.

At this point however, Simulation2, when run, throws IllegalArgumentException: Could not locate feeder file, which is puzzling to see, because the type of csv(“target/file.csv”).circular is SourceFeederBuilder, which is a FeederBuilder, which is a () => Feeder[Any], and that is a function, exactly what you want: something that does not get evaluated at construction time.

After some scrambling, i created a custom feeder where i could control for my sanity that was in fact a () => Feeder[Any].
Passing my custom feeder to the feed() method worked, but that custom feeder code can actually be simplified to one line, after noticing that .circular is a macro, and that’s why it was getting prematurely evaluated.
So, to resolve that, instead of passing:
csv(“target/file.csv”).circular

you must pass:
() => csv(“target/file.csv”).circular.apply()

as a feeder for the scenario in Simulation2, which does the same as my custom feeder above. The scenario must be written as:

val scn = scenario(“sim2”)
.feed( () => csv(“target/file.csv”).circular.apply() )
.exec(…)

When class Simulation2 is loaded, it will not complain about file target/file.csv missing, it will run the before{} clause that saves file file.csv, and finally, when the setUp method will be called, the feeder will be run correctly.

(Again, note that if you pass just csv(“myfile.csv”) to feed(), without calling circular or shuffle on it, it will work fine. It is only when you pass it as csv(“file.csv”).circular or csv(“file.csv”).shuffle or csv(“file.csv”).random, etc, that will not work: the AST at that point is already modified)

In the end i chose this approach over the two possibilities you list above: option #1 i believe will not work, because the Iterator (the Feeder) is already run at that point. And option #2, in general, i always tend to avoid, just because i don’t like to run logic in constructors and it is also more readable in a before{} method, i think, but that’s just my own practice, because it would work, of course.

Max