Feeders

[Matt_Royer]

Hi,

In the documentation for Gatling it says: “A Feeder is an object shared amongst users that will inject data into a given user’s Session every time it reaches a feed method call.”

Is there anything in Gatling that will allow me to have multiple users, but also very specific data for those users to post on my site? This way, the data in a second feeder will only pertain to the user that feeder specifies.

For example, if I have a user_credentials.csv that has two columns of username and password.

But then I have another csv, we’ll call it documents.csv. In this csv file, we can have whatever parameters my site needs for a user to do something with a document. But the first column will be username.

This way I can have user “mroyer” for however many rows I need. Where Gatling can only use that user’s data for site POSTs.

So it will be formatted as “mroyer” first column for 3 rows. This indicates three documents for mroyer. And the other columns can have whatever POST data I need for that user.

Sorry if I’m rambling. I’m just trying to get the point across and hope I’m doing an okay job.

Any help would be greatly appreciated.

[Excilys]

Hi,

You’ll have to do it programmatically: load the documents.csv fil into a Map.
I’ll show you tomorrow.

Stéphane

[Excilys]

This was an interesting exercice, so I documented it in the Cook Book: https://github.com/excilys/gatling/wiki/Inject-User-Specific-Data

Cheers,

Stéphane

[Matt_Royer]

Thank you so much Stéphane!

That is such a great approach. I’m excited to implement this.

Thanks again for all your hard work. You’re always so quick with your answers.

I truly appreciate this.

–Matt Royer

[Excilys]

You’re welcome.

This one was fun.

[Matt_Royer]

Most likely I’m just a moron, but I’m getting an error when trying to run my scenario. I’m sure I’m doing something WAY off.

Here’s the error:

11:12:50.945 [WARN ] c.e.e.g.c.a.WhileAction - Condition evaluation crashed, exiting loop
java.lang.IllegalArgumentException: No Matching Session attribute for key [Lscala.collection.immutable.Map;@399af260
at com.excilys.ebi.gatling.core.session.Session$$anonfun$getTypedAttribute$1.apply(Session.scala:72) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.session.Session$$anonfun$getTypedAttribute$1.apply(Session.scala:72) ~[gatling-core-1.4.4.jar:na]
at scala.Option.getOrElse(Option.scala:108) ~[scala-library-2.9.3.jar:na]
at com.excilys.ebi.gatling.core.session.Session.getTypedAttribute(Session.scala:72) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$class.seq$1(Loops.scala:81) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$class.continueCondition$1(Loops.scala:91) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$$anonfun$foreach$1.apply(Loops.scala:94) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$$anonfun$foreach$1.apply(Loops.scala:94) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.action.WhileAction.execute(WhileAction.scala:56) [gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.action.Action$$anonfun$receive$1.apply(Action.scala:31) [gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.action.Action$$anonfun$receive$1.apply(Action.scala:30) [gatling-core-1.4.4.jar:na]
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:76) [scala-library-2.9.3.jar:na]
at akka.actor.Actor$class.apply(Actor.scala:318) [akka-actor-2.0.4.jar:2.0.4]
at com.excilys.ebi.gatling.core.action.WhileAction.apply(WhileAction.scala:31) [gatling-core-1.4.4.jar:na]
at akka.actor.ActorCell.invoke(ActorCell.scala:626) [akka-actor-2.0.4.jar:2.0.4]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197) [akka-actor-2.0.4.jar:2.0.4]
at akka.dispatch.Mailbox.run(Mailbox.scala:179) [akka-actor-2.0.4.jar:2.0.4]
at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:516) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
[akka-actor-2.0.4.jar:2.0.4]

And here is my code:

package Multi
import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import com.excilys.ebi.gatling.jdbc.Predef._
import com.excilys.ebi.gatling.http.Headers.Names._
import akka.util.duration._
import bootstrap._

class MultiUser extends Simulation {

val httpConf = httpConfig
.baseURL(“http://load02.lab”)
.acceptHeader(“text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8”)
.acceptCharsetHeader(“ISO-8859-1,utf-8;q=0.7,*;q=0.3”)
.acceptLanguageHeader(“en-US,en;q=0.8”)
.acceptEncodingHeader(“gzip,deflate,sdch”)
.userAgentHeader(“Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.97 Safari/537.11”)

val headers_1 = Map(
"Cache-Control" → “”“max-age=0"”"
)

val headers_12 = Map(
"Cache-Control" → “”“max-age=0"”",
"Content-Type" → “”“application/x-www-form-urlencoded”"",
"Origin" → “”“http://load02.lab”""
)

val headers_13 = Map(
"Accept" → “”"/"""
)

val credentials = csv(“Multi_user_credentials.csv”)
val documents = csv(“Multi_documents.csv”).groupBy(_(“username”))

val injectUserDocuments = (session: Session) => {
val username: String = session.getTypedAttribute(“username”)
session.setAttribute(“documents”, documents(username))
}

val injectDocumentData = (session: Session) => {
val documentData: Map[String, String] = session.getTypedAttribute(“document”)
session.setAttributes(documentData)
}

val scn = scenario(“Docs”)
.feed(credentials)
.exec(injectUserDocuments)

.exec(http(“Login”)
.post("/presentation/login/index.cfm")
.headers(headers_12)
.queryParam(""“initialRequest”"", “”"/presentation/index.cfm""")
.queryParam(""“action”"", “”“login”"")
.param(""“username”"", “${username}”)
.param(""“password”"", “${password}”)
.param(""“initialRequest”"", “”"/presentation/index.cfm""")
)
.foreach("${documents}", “document”) {
exec(injectDocumentData)

.exec(http(“List Task”)
.get("//main/portal/lists/listDetail.cfm")
.queryParam(""“var1"”", “${var1}”)
.queryParam(""“var2"”", “${var2}”)
.queryParam(""“var3"”", “${var3}”)
.queryParam(""“var4"”", “${var4}”)
.queryParam(""“var5"”", “${var5}”)
.queryParam(""“var6"”", “${var6}”)
.queryParam(""“var7"”", “${var7}”)
.queryParam(""“var8"”", “${var8}”)
)

.exec(http(“Popup Approve Task”)
.get("//Main/Approval/Packets/approve_pop_up.cfm")
.queryParam(""“PacketID”"", “${var2}”)
.queryParam(""“PacketName”"", “${var3}”)
.queryParam(""“PacketStep”"", “${var4}”)
.queryParam(""“alt_user”"", “${var5}”)
.queryParam(""“parallelID”"", “${var8}”)
)
}

setUp(scn.users(4).protocolConfig(httpConf))
}

If you need the contents of my csv files, let me know. They aren’t long.

Also, I converted this script to the 1.4.4 version of Gatling. I am coming from Gatling 1.3. So I could have converted it all wrong as well. Completely possible. Before converting, this was working with one user (without loops) with the values statically placed into the queryParams, so I know my site paths are good. I upgraded to 1.4.4 because I saw in the Gatling documentation that .foreach was added there.

Thanks for any help you can give.

–Matt Royer

[Excilys]

That’s actually my fault: I’ve written the sample against Gatling 2 without realizing that the foreach loop doesn’ t work exactly the same way due to limitations in Gatling 1 EL.
So foreach first parameter in Gatling 1 is not an EL but the String key of the sequence and you have to write foreach(“documents”, “document”).

Let me know if that works, now.

Cheers,

Stéphane

[Matt_Royer]

I changed it to .foreach(“documents”, “document”) and I still get the same error.

I do have a question though. And I don’t know if this is messing it up. On the section where you put:

val injectDocumentData = (session: Session) => {
val documentData: Map[String, String] = session.getTypedAttribute(“document”)
session.setAttributes(documentData)
}

The piece that says session.getTypedAttribute(“document”)… Is that trying to pull a document column from the csv file? If so, I don’t have that column, but I can put one in if it will make this run.

Thanks again,

–Matt Royer

[Matt_Royer]

Oops. I thought it was the same error. It’s a different one:

15:26:15.568 [WARN ] c.e.e.g.c.a.WhileAction - Condition evaluation crashed, exiting loop
java.lang.ClassCastException: [Lscala.collection.immutable.Map; cannot be cast to scala.collection.Seq
at com.excilys.ebi.gatling.core.structure.Loops$class.seq$1(Loops.scala:81) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$class.continueCondition$1(Loops.scala:91) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$$anonfun$foreach$1.apply(Loops.scala:94) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.structure.Loops$$anonfun$foreach$1.apply(Loops.scala:94) ~[gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.action.WhileAction.execute(WhileAction.scala:56) [gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.action.Action$$anonfun$receive$1.apply(Action.scala:31) [gatling-core-1.4.4.jar:na]
at com.excilys.ebi.gatling.core.action.Action$$anonfun$receive$1.apply(Action.scala:30) [gatling-core-1.4.4.jar:na]
at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:76) [scala-library-2.9.3.jar:na]
at akka.actor.Actor$class.apply(Actor.scala:318) [akka-actor-2.0.4.jar:2.0.4]
at com.excilys.ebi.gatling.core.action.WhileAction.apply(WhileAction.scala:31) [gatling-core-1.4.4.jar:na]
at akka.actor.ActorCell.invoke(ActorCell.scala:626) [akka-actor-2.0.4.jar:2.0.4]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:197) [akka-actor-2.0.4.jar:2.0.4]
at akka.dispatch.Mailbox.run(Mailbox.scala:179) [akka-actor-2.0.4.jar:2.0.4]
at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:516) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1479) [akka-actor-2.0.4.jar:2.0.4]
at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
[akka-actor-2.0.4.jar:2.0.4]

[Excilys]

My fault, again.
foreach expects a Seq, while documents(username) is an Array (just like in Java, an Array is not a Collection), so one just have to convert it into a Seq.
I updated the wiki page, see the .toSeq I added.

I tested with Gatling 1.4, so I’m sure it works now…

Have fun,

Stéphane

[Excilys]

Ah, and regarding your question, that’s not how it works:

1. csv(“documents.csv”) => read all the csv file and load it into an array of “records” (maps where keys are the column names)

2. groupBy(_(“username”) => group the array entries by “username” map entry

3. injectUserDocuments => inject into the user session the Sequence of maps whose “username” entry equals the value injected by the credentials feeder

4. foreach => loop on the “documents” sequence previously stored in the session

5. injectDocumentData => inject into the user session the document entries (the document is a map, set its entries directly as attributes)

[Matt_Royer]

That’s the part I’m trying to get right and keep messing up. In my document.csv I have it setup like so:

username,var1,var2,var3,var4,var5,var6,var7,var8
lt_user1,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user1,loadTask,D2EB06520E44204DB4,Task2,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user2,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B

lt_user3,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user3,loadTask,EF38FA34044EA04A02,Task5,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B

In my script, can I use it like so?

.foreach(“documents”, “document”) {
exec(injectDocumentData)
.exec(http(“List Task”)
.get("//main/portal/lists/listDetail.cfm")
.queryParam(""“var1"”", “${var1}”)
.queryParam(""“var2"”", “${var2}”)
.queryParam(""“var3"”", “${var3}”)
.queryParam(""“var4"”", “${var4}”)
.queryParam(""“var5"”", “${var5}”)
.queryParam(""“var6"”", “${var6}”)
.queryParam(""“var7"”", “${var7}”)
.queryParam(""“var8"”", “${var8}”)
)
}

Where the first part of the queryParam needs to be var1-8 because of how our site takes the attributes. But can I take use ${var1}-${var8} as the second queryParam? Or how would I set these var(n)'s to what I’m injecting into the session?

I hope that makes sense. I’m still learning. Sorry.

–Matt Royer

[Excilys]

I don’t really understand what you’re trying to achieve.
For example, with the first record, “${var1}-${var8}” will be resolved as “**loadTask-**A52EB3FEC063418D8B”

[Matt_Royer]

Sorry, I put that to mean ${var1} through ${var8}. I didn’t want the hyphen between them.

I mainly want to see if I can use the script as I have it here:

[Excilys]

So, yes, exactly

[Matt_Royer]

Sweet!!! Thanks for all your help.

I ran it and requests were sent successfully. But for my Login request, it only did one, and for my other request (List Tasks), it did two. It looks like it’s only doing the first user’s information from the document.csv. Which is great that it’s working on the first one! I just don’t know how to loop over it for each user that is in both the credentials.csv and document.csv.

Thanks again Stéphane.

[Excilys]

What do you mean by “it only did one”?
The default feeder strategy is queue, so your credentials feeder will keep on providing data as long as it doesn’t run out of records. How many records do you have in your file.
Depending on your use case, you might want to use another strategy, such as circular or random. See feeder docu.

[Matt_Royer]

Here are my records from the document.csv:

username,var1,var2,var3,var4,var5,var6,var7,var8
lt_user1,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user1,loadTask,D2EB06520E44204DB4,Task2,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user2,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user2,loadTask,D7C62E1C0AE4804714,Task3,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user2,loadTask,65E068370D7AD0444C,Task4,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user3,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user3,loadTask,EF38FA34044EA04A02,Task5,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user4,loadTask,6FC3663300B0204712,Task1,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user4,loadTask,D2EB06520E44204DB4,Task2,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user4,loadTask,D7C62E1C0AE4804714,Task3,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user4,loadTask,65E068370D7AD0444C,Task4,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B
lt_user4,loadTask,EF38FA34044EA04A02,Task5,A52EB3FEC063418D8B,1,module:docs,A52EB3FEC063418D8B

And here are my records from the credentials.csv:

username,password
lt_user1,1
lt_user2,1
lt_user3,1
lt_user4,1

And here are my results from running the test:

---- Sign Off ------------------------------------------------------------------

Users : [#################################################################]100%

waiting:0 / running:0 / done:1
---- Requests ------------------------------------------------------------------

> Login OK=1 KO=0

> Login Redirect 1 OK=1 KO=0

> List Task OK=2 KO=0

[Excilys]

Are you sure you still have 4 users in the setUp?
Could you post a gist of your simulation, please?

[Matt_Royer]

I am the biggest moron!

I put it back to one user while I was getting those errors. That way I didn’t see so many errors in the console. As soon as your post came through, I knew I messed up. I looked at the rest of my file so many times, but thought I still had users set to 4.

I’m sorry to waste your time. And I am SO grateful for all of your help.

–Matt Royer