# Feeder file generation from a Gatling script?
**URL:** https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605
**Category:** Gatling (Open-Source)
**Created:** [October 29, 2014, 7:44am UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605 "2014-10-29T07:44:24Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [October 29, 2014, 7:44am UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/1 "2014-10-29T07:44:24Z")
I’d like to find out if it is possible to _write_ a Feeder file using a Gatling script.
I am using Gatling to test a RESTful API that offers methods for Users to create objects in the target system.
Each object created by a user has a GUID that subsequently becomes associated with that user’s ID number for a rather long time (6 mo).
I have a csv file of unique userIds as well as a test data generation script that can create objects for Users. However, the objectIDS are not currently captured; they go straight to the database. This is good for the service being tested, but not so good for me.
What I would like to do is to have my script capture the GUID as soon as it is created and somehow link it to the owning UserId so I can use both values for later tests. Out of the UserIds I have, a few might get used twice in a single pass of the script, but the majority will get just one objectID per script run.
Since these objects are likely to expire before the Users do, I would like to keep my User file intact and have this new file be available “on demand”.
What would I need to do to implement something like this?
---
### Author:  [@Excilys](https://community.gatling.io/u/Excilys)
#### Post date: [October 30, 2014, 2:06pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/2 "2014-10-30T14:06:35Z")
Why do you want it it be a file? Why not a ConcurrentHashMap?
---
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [October 30, 2014, 7:47pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/3 "2014-10-30T19:47:51Z")
I was thinking of using a file because that is something I already know how to do, and it
fits in well with the current implementation patterns I’ve been using.
I was also concerned about resource usage, because lots of objects will be created during this test (upwards of 10k during any given test run).
If resources are no concern, I would certainly look into it deeper.
What would be the advantages for use with Gatling?
---
### Author:  [@jarrowwx](https://community.gatling.io/u/jarrowwx)
#### Post date: [October 30, 2014, 8:04pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/4 "2014-10-30T20:04:34Z")
Are you needing this cache of GUIDS to live between runs of Gatling? If no, then just throw memory at the problem and do it in RAM.
If you need the IDs to be persisted and accessed in future runs, then a file or a database is the right thing to do.
HOWEVER: if you need both, you need a hybrid solution. Because writing it to the file does not make it immediately available for a feeder, because feeders load everything into memory at once (most of the time, I believe).
---
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [October 31, 2014, 2:59pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/5 "2014-10-31T14:59:29Z")
The values will be used by future runs, so it looks like some kind of persistent storage is the best way to go.
I found some forum posts that indicate I could access the GUIDS by using a Session function, but I’m a little mystified about how to get them from Gatling Session to a file.
---
### Author:  [@jarrowwx](https://community.gatling.io/u/jarrowwx)
#### Post date: [October 31, 2014, 3:57pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/6 "2014-10-31T15:57:32Z")
In your simulation, before you define your scenario, you would open the file, something like this:
`
val guid\_data = {
val fos = new java.io.FileOutputStream( “guid.csv” )
val pw = new java.io.PrintWriter( fos, true )
pw.println( “list,of,column,headers” )
pw
}
`
In the simulation, you would need to write to the file, like
`guid_data.println( ... )`
However, in this simplified model, you don’t mix reads and writes. You run a scenario that generates the file, and a different scenario would depend on the data in the file.
If you need to mix reads and writes (like, I need the GUID, and if I get an error that says the GUID has expired, then get a new one and save it for next time), then you need to use a database, or use files on the filesystem as a cache.
---
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [November 1, 2014, 2:56am UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/7 "2014-11-01T02:56:40Z")
Ah…ok. Now the lights are coming on!
So, this looks likes pretty regular code. The biggest “gotcha” is putting it in the right place to execute.
I have a script that is dedicated for generating test data to be used by other Simulations, so this will work well for my use case.
Thank you very much.
---
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [January 11, 2015, 8:23pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/8 "2015-01-11T20:23:38Z")
@Stephane I’m revisiting this topic because now I actually have a chance to work on it 🙂
I was thinking of putting the values into a CSV file so I could capture the data and retain it for multiple test runs. The associations I intend to capture are stored in multiple databases, so creating this file will help us test just this one microservice in isolation.
However, I am curious about the ConcurrentHashMap. Are there any examples of this in the Gatling docs or on GitHub where I could see some implementations? Also, what would be the advantages over using a feeder file?
---
### Author:  [@Excilys](https://community.gatling.io/u/Excilys)
#### Post date: [January 13, 2015, 3:29pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/9 "2015-01-13T15:29:34Z")
I was only suggesting ConcurrentHashMap if you didn’t need to retain the data between runs (like only storing in memory to share some data between scenarios).
Depending on how much you want to store on disk and how fast, you might want to properly buffer, or even retain everything in memory in a ConcurrentQueue, and only dump on file in the “after” hook.
Cheers,
Stéphane
---
### Author:  [@nadine.whitfield](https://community.gatling.io/u/nadine.whitfield)
#### Post date: [February 10, 2015, 6:48pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/10 "2015-02-10T18:48:08Z")
Ah, ok.
Well, I got the script to work beautifully. I can tell it how many rows to create, and it writes the CSV file just fine. Script consists of 7 requests that are chained together and share various attributes from the session object.
I now have a requirement to run this Scenario multiple times for a _single_ virtual user. The number of times to repeat has to be a random Integer between 1 and some random max value that is provided at runtime (in this case from Jenkins).
I thought I could accomplish this by wrapping the entire Scenario inside a repeat block, but now I’m getting a runtime error from Gatling with text like this\>
```auto
[ERROR] [02/10/2015 18:16:41.450] [GatlingSystem-akka.actor.default-dispatcher-2] [akka://GatlingSystem/user/$a] requirement failed: Scenario CreateFullDataSet is empty
java.lang.IllegalArgumentException: requirement failed: Scenario CreateFullDataSet is empty
at scala.Predef$.require(Predef.scala:233)
at io.gatling.core.scenario.Simulation$$anonfun$scenarios$2.apply(Simulation.scala:40)
at io.gatling.core.scenario.Simulation$$anonfun$scenarios$2.apply(Simulation.scala:40)
at scala.collection.immutable.List.foreach(List.scala:318)
at io.gatling.core.scenario.Simulation.scenarios(Simulation.scala:40)
at io.gatling.core.controller.Controller$$anonfun$1.applyOrElse(Controller.scala:80)
at akka.actor.Actor$class.aroundReceive(Actor.scala:465)
at io.gatling.core.akka.BaseActor.aroundReceive(BaseActor.scala:23)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
```
What would be the best way to fix this error?
Basic stripped layout of my code is below. My attempted changes are in Red.
val scnMyFullScenario = scenario(“CreateFullDataSet”)
// generate random number to set number of repeats
val start = 1
val rnd = new scala.util.Random
val hits = start + rnd.nextInt( (maxRepeatsPerUser - start) + 1 )
repeat( hits ) {
feed(senderFeeder)
.feed(receiverFeeder)
.feed(traceIdFeeder)
.feed(timestamp)
.feed(actionObjectFeeder)
// place part of payload into Session for further processing
.exec( session =\> { // doing stuff with session variables here
session })
.exec( http(“Request1”))).asJSON
.check(status.is(200)
, jsonPath("$.ids").saveAs(“PostId”))
)
.pause(50 milliseconds)
// remove unwanted characters from PostId
.exec(session =\> { // more session work
session })
.exec( http(“Request2” ))).asJSON
.check(status.is(204))
)
.exec(http(“Request3”))).asJSON
.check(status.is(204))
)
.exec(http(“Request4”))).asJSON
.check(status.is(204))
)
.exec(http(“Request5”))).asJSON
.check(status.is(204))
)
.exec(
http(“Request6”))).asJSON
.check(status.is(204))
)
.exec( http(“Request7”) )).asJSON
.check(status.is(200),
jsonPath("$.id").saveAs(“CommentId”))
)
// remove unwanted characters from CommentId
.exec(session =\> { // doing stuff with session attributes
session })
// extract values from Session and write to file
.exec(session =\> { // extract attributes and set them up to be written in CSV format
session })
}
setUp(
scnMyFullScenario.inject(rampUsers(numberRows) over (runTime seconds))
.protocols(httpProtocol)
)
---
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [February 17, 2015, 3:21pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/11 "2015-02-17T15:21:51Z")
Problem fixed. I had the right bits, but in the wrong order.
The biggest problem was the placement of the repeat block. I moved this a couple of lines later so the feed statements could set up a SenderId and ReceiverId first. Then I used the repeat block to wrap the following feed statements and subsequent code that creates a post, manipulates session attributes and writes the line to a file.
Then of course, it is important to be extra aware of placement of the _dot operator._ I have not seen any formal documentation of this, but have noticed a correlation between placement of dot operators and sections of block code like .group, .repeat, .asLongAs, etc - the first statement after the opening has to have the dot operator removed.
After working on this problem, I’ve learned to make dot operators one of the first places I look when there are problems.
---
### Author:  [@Srinivas\_D](https://community.gatling.io/u/Srinivas_D)
#### Post date: [October 7, 2016, 12:44pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/12 "2016-10-07T12:44:46Z")
Hello [nadine.w...@nike.com](mailto:nadine.w...@nike.com)
could you post the code how the session attributes are extracted and written to csv format
ref in the section
// extract values from Session and write to file
.exec(session =\> { // extract attributes and set them up to be written in CSV format
session })
Would of great help.
Regards
---
### Author:  [@Nicholas\_Ling](https://community.gatling.io/u/Nicholas_Ling)
#### Post date: [November 18, 2016, 2:19pm UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/13 "2016-11-18T14:19:36Z")
please. It’s lack of documentation.
---
### Author:  [@Nadine\_Whitfield](https://community.gatling.io/u/Nadine_Whitfield)
#### Post date: [May 23, 2017, 5:23am UTC](https://community.gatling.io/t/feeder-file-generation-from-a-gatling-script/1605/14 "2017-05-23T05:23:42Z")
Hi Srinivas, Nicholas Ling -
So sorry! I’ve been off the mailing list for this group and just found your questions.
You probably figured it out by now, but I did want respond.
The solution I came up with uses the Gatling session (which is essentially a giant Map structure) to capture the values.
Session is then passed to a custom Scala function that extracts data and forms it into a comma-delimited row of data that can be written to a file.
Code is here on Github\> [https://gist.github.com/infomaven/1d9393c83d57eef7ae72e2b2818be718#file-gatlinggeneratedfeeder-scala](https://gist.github.com/infomaven/1d9393c83d57eef7ae72e2b2818be718#file-gatlinggeneratedfeeder-scala)
_Disclaimer - this code has only been tested with Gatling 2.0._
_Some of the syntax might be different now, but the basic concepts should not change very much._
Regards.
- infomaven