Grab all feeder records in 3.0.1

Hi Stephane et team,

While trying out our simulator with Gatling 3.0.1, we found another unexpected change in behavior of our feeders using readRecords:
https://gatling.io/docs/current/session/feeder/#grabbing-records

This is the definition of one of our feeders:

`

val simDataTokens: Vector[Map[String, String]] = {
try {
csv(file).readRecords.as[Vector[Map[String, String]]]
.zipWithIndex
.withFilter(r => {
r._1.contains(“name”) &&
r._1.contains(“token”)
})
.map(r => {
val id = r._2.toString
val tokens = r._1

// DEBUG
println(">>> id: " + id + ", tokens: " + tokens)

tokens + (“id” → id)
})
} catch {
case _: Exception => Vector.empty
}
}

// DEBUG
println(">>> simDataTokens records: " + simDataTokens.size)

`

With Gatling 2.3.1 (using csv(file).**records.**toVector instead), simDataTokens returned three expected records and size = 3 with following debugging info:

`

GATLING_HOME is set to /home/gatling/2.3.1

id: 0, tokens: Map(name → a, token → 1)
id: 1, tokens: Map(name → b, token → 2)
id: 2, tokens: Map(name → c, token → 3)
simDataTokens records: 3

10:49:44.179 [INFO ] i.g.h.a.HttpEngine - Start warm up
10:49:44.287 [INFO ] i.g.h.a.HttpEngine - Warm up done
Simulation com.illumio.agentSimulation started…

`

With Gatling 3.0.1, simDataTokens only returned the first record, and its size becomes 0. This is unexpected:

`

GATLING_HOME is set to /home/gatling/3.0.1

id: 0, tokens: Map(name → a, token → 1)
simDataTokens records: 0
Simulation com.illumio.agentSimulation started…

`

Thank you for your help along the way.

Vu

Nah, your code is broken:

.withFilter(r => {
r._1.contains(“name”) &&
r._1.contains(“token”)
})

How do you expect r._1 to both contain “name” and “token”? :stuck_out_tongue:

Because with zipWithIndex, r._1 is a Map[String, String] (which are the records from csv(file)).

These codes have worked for us for years using Gatling 2.3.1 and Gatling 2.2.0. They only failed when we ran them with Gatling 3.0.1.

(BTW, the debug outputs were generated earlier today).

Vu

My2cents: there’s an exception and you don’t see it because you silently trap it and return Vector.empty, which looks very fishy to me.
Have you checked the migration guide and how feeder file location resolution has changed (“data” folder is dropped)?

Much appreciation for Your2cents, Stephane.

I repeated my experiment with following trimmed down feeder (with line number for reference):

`

200 val simDataTokens: Vector[Map[String, Any]] = {
201 csv(simTokens).readRecords.as[Vector[Map[String, String]]]
202 .zipWithIndex
203 .map(r => {
204 val id = r._2
205 val tokens = r._1
206
207 // DEBUG
208 println(">>> id: " + id + “, tokens: " + tokens)
209
210 tokens + (“id” → id)
211 })
212 }
213
214 println(”>>> simDataTokens records: " + simDataTokens.size)

`

And there was indeed an exception:

`

GATLING_HOME is set to /home/gatling/3.0.1

id: 0, tokens: Map(name → a, token → 1)
15:04:51.395 [ERROR] i.g.a.Gatling$ - Run crashed
java.lang.UnsupportedOperationException: null
at io.gatling.core.feeder.ArrayBasedMap.$plus(ArrayBasedMap.scala:28)
at io.gatling.core.feeder.ArrayBasedMap.$plus(ArrayBasedMap.scala:26)
at com.illumio.commonHeader$.$anonfun$simDataTokens$1(commonHeader.scala:210)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:233)
at scala.collection.Iterator.foreach(Iterator.scala:937)
at scala.collection.Iterator.foreach$(Iterator.scala:937)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1425)
at scala.collection.IterableLike.foreach(IterableLike.scala:70)
at scala.collection.IterableLike.foreach$(IterableLike.scala:69)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at scala.collection.TraversableLike.map(TraversableLike.scala:233)
at scala.collection.TraversableLike.map$(TraversableLike.scala:226)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at com.illumio.commonHeader$.(commonHeader.scala:203)
at com.illumio.commonHeader$.(commonHeader.scala)
… 14 common frames omitted

`

It appears the Map addition operation (tokens + (“id” → id)) is not supported? Any suggestion if that’s the case?
https://github.com/gatling/gatling/blob/b7617bb05e895558da9c71c70df13c6db34d8951/gatling-core/src/main/scala/io/gatling/core/feeder/ArrayBasedMap.scala#L28

Thank you again, Stephane.

Vu

Our ArrayBasedMap doesn’t currently implement +, - and updated operations, simply because we don’t need them internally in Gatling and use cases such as yours are pretty exotic.
Feel free to send a PR to implement them.

https://github.com/gatling/gatling/issues/3625

Thank you.