ConcurrentModificationException

Hello,

I recently switched from 2.1.3 to 2.1.7 and am now seeing the “java.util.ConcurrentModificationException” below showing up in my console output in Jenkins. The test runs fine for about 30 to 40 minutes (40 to 60 repeated iterations) and then this error pops up. Once the error is seen the first time, the “i.g.c.r.w.FileDataWriter - Can’t handle RequestMessage” starts happening on all subsequent requests until the test ends. If it makes any difference, the first error always seems to happen on the login step, the only step that has a URL redirect.

20:58:14.433 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 01A uri=http://MyServer:8888/BasePath/auth: scenario=customer_1, userId=1234-0
20:58:14.444 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 02A uri=http://MyServer:8888/BasePath/auth/jsRoutes/authRoutes: scenario=customer_1, userId=1234-0
20:58:15.445 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 03A uri=http://MyServer:8888/BasePath/auth: scenario=customer_1, userId=1234-0
20:58:16.011 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 04A uri=http://MyServer:8888/BasePath/auth_token: scenario=customer_1, userId=1234-0
20:58:16.103 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 04A uri=http://MyServer:8888/BasePath/launcher: scenario=customer_1, userId=1234-0
[ERROR] [12/03/2015 20:58:16.116] [GatlingSystem-akka.actor.default-dispatcher-13] [akka://GatlingSystem/user/FileDataWriter] null
java.util.ConcurrentModificationException
at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394)
at java.util.LinkedHashMap$ValueIterator.next(LinkedHashMap.java:409)
at java.util.AbstractCollection.addAll(AbstractCollection.java:341)
at java.util.LinkedHashSet.(LinkedHashSet.java:169)
at com.ning.http.client.FluentCaseInsensitiveStringsMap.keySet(FluentCaseInsensitiveStringsMap.java:341)
at com.ning.http.client.RequestBuilderBase$RequestImpl.toString(RequestBuilderBase.java:258)
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$$anonfun$io$gatling$core$result$writer$FileDataWriter$RequestMessageSerializer$$serializeExtraInfo$extension$1.apply(FileDataWriter.scala:67)
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$$anonfun$io$gatling$core$result$writer$FileDataWriter$RequestMessageSerializer$$serializeExtraInfo$extension$1.apply(FileDataWriter.scala:67)
at scala.collection.immutable.List.map(List.scala:273)
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$.io$gatling$core$result$writer$FileDataWriter$RequestMessageSerializer$$serializeExtraInfo$extension(FileDataWriter.scala:67)
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$.serialize$extension(FileDataWriter.scala:78)
at io.gatling.core.result.writer.FileDataWriter.onRequestMessage(FileDataWriter.scala:140)
at io.gatling.core.result.writer.DataWriter$$anonfun$initialized$1.applyOrElse(DataWriter.scala:104)
at akka.actor.Actor$class.aroundReceive(Actor.scala:467)
at io.gatling.core.akka.BaseActor.aroundReceive(BaseActor.scala:22)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)
at akka.actor.ActorCell.invoke(ActorCell.scala:487)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)
at akka.dispatch.Mailbox.run(Mailbox.scala:220)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

20:58:16.125 [ERROR] i.g.c.r.w.FileDataWriter - Actor io.gatling.core.result.writer.FileDataWriter@2c651db0 crashed on message Some(RequestMessage(customer_1,1234-0,List(),Login 04A,1449176296011,1449176296011,1449176296101,1449176296101,OK,None,List(http://MyServer:8888/BasePath/auth_token POST headers: Referer:http://MyServer:8888/BasePath/auth Origin:http://MyServer:8888 Connection:keep-alive Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8 Cache-Control:max-age=0 Accept-Language:en-US,en;q=0.8 Accept-Encoding:gzip, deflate User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36 formParams: SecureTicket: RedirectUrl:http://MyServer:8888/BasePath/launcher)))
java.util.ConcurrentModificationException: null
at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(LinkedHashMap.java:394) ~[na:1.7.0_79]
at java.util.LinkedHashMap$ValueIterator.next(LinkedHashMap.java:409) ~[na:1.7.0_79]
at java.util.AbstractCollection.addAll(AbstractCollection.java:341) ~[na:1.7.0_79]
at java.util.LinkedHashSet.(LinkedHashSet.java:169) ~[na:1.7.0_79]
at com.ning.http.client.FluentCaseInsensitiveStringsMap.keySet(FluentCaseInsensitiveStringsMap.java:341) ~[async-http-client-1.9.30.jar:na]
at com.ning.http.client.RequestBuilderBase$RequestImpl.toString(RequestBuilderBase.java:258) ~[async-http-client-1.9.30.jar:na]
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$$anonfun$io$gatling$core$result$writer$FileDataWriter$RequestMessageSerializer$$serializeExtraInfo$extension$1.apply(FileDataWriter.scala:67) ~[gatling-core-2.1.7.jar:2.1.7]
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$$anonfun$io$gatling$core$result$writer$FileDataWriter$RequestMessageSerializer$$serializeExtraInfo$extension$1.apply(FileDataWriter.scala:67) ~[gatling-core-2.1.7.jar:2.1.7]
at scala.collection.immutable.List.map(List.scala:273) ~[scala-library-2.11.7.jar:na]
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$.io$gatling$core$result$writer$FileDataWriter$RequestMessageSerializer$$serializeExtraInfo$extension(FileDataWriter.scala:67) ~[gatling-core-2.1.7.jar:2.1.7]
at io.gatling.core.result.writer.FileDataWriter$RequestMessageSerializer$.serialize$extension(FileDataWriter.scala:78) ~[gatling-core-2.1.7.jar:2.1.7]
at io.gatling.core.result.writer.FileDataWriter.onRequestMessage(FileDataWriter.scala:140) ~[gatling-core-2.1.7.jar:2.1.7]
at io.gatling.core.result.writer.DataWriter$$anonfun$initialized$1.applyOrElse(DataWriter.scala:104) ~[gatling-core-2.1.7.jar:2.1.7]
at akka.actor.Actor$class.aroundReceive(Actor.scala:467) ~[akka-actor_2.11-2.3.12.jar:na]
at io.gatling.core.akka.BaseActor.aroundReceive(BaseActor.scala:22) ~[gatling-core-2.1.7.jar:2.1.7]
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516) ~[akka-actor_2.11-2.3.12.jar:na]
at akka.actor.ActorCell.invoke(ActorCell.scala:487) ~[akka-actor_2.11-2.3.12.jar:na]
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238) ~[akka-actor_2.11-2.3.12.jar:na]
at akka.dispatch.Mailbox.run(Mailbox.scala:220) ~[akka-actor_2.11-2.3.12.jar:na]
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397) [akka-actor_2.11-2.3.12.jar:na]
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.11.7.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.11.7.jar:na]
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.11.7.jar:na]
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.11.7.jar:na]
20:58:16.134 [ERROR] i.g.c.r.w.FileDataWriter - Can’t handle RequestMessage(customer_1,1234-0,List(),Login 04A Redirect 1,1449176296103,1449176296103,1449176296132,1449176296132,OK,None,List(http://MyServer:8888/BasePath/launcher GET headers: Referer:http://MyServer:8888/BasePath/auth Origin:http://MyServer:8888 Connection:keep-alive Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8 Cache-Control:max-age=0 Accept-Language:en-US,en;q=0.8 Accept-Encoding:gzip, deflate User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36)) when in uninitialized state, discarding
20:58:16.134 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 05A uri=http://MyServer:8888/BasePath/launcher: scenario=customer_1, userId=1234-0
20:58:16.164 [ERROR] i.g.c.r.w.FileDataWriter - Can’t handle RequestMessage(customer_1,1234-0,List(),Login 05A,1449176296134,1449176296135,1449176296162,1449176296162,OK,None,List(http://MyServer:8888/BasePath/launcher GET headers: Referer:http://MyServer:8888/BasePath/auth Connection:keep-alive Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8 Cache-Control:max-age=0 Accept-Language:en-US,en;q=0.8 Accept-Encoding:gzip, deflate User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36 Host:http://MyServer:8888)) when in uninitialized state, discarding
20:58:16.165 [INFO ] i.g.h.a.HttpEngine - Sending request=Login 06A uri=http://MyServer:8888/BasePath/auth/jsRoutes/authRoutes: scenario=customer_1, userId=1234-0
20:58:16.171 [ERROR] i.g.c.r.w.FileDataWriter - Can’t handle RequestMessage(customer_1,1234-0,List(),Login 06A,1449176296165,1449176296165,1449176296170,1449176296170,OK,None,List(http://MyServer:8888/BasePath/auth/jsRoutes/authRoutes GET headers: Referer:http://MyServer:8888/BasePath/launcher Connection:keep-alive Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8 Accept-Language:en-US,en;q=0.8 Accept-Encoding:gzip, deflate User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36 Host:http://MyServer:8888)) when in uninitialized state, discarding

Any ideas as to what might be happening here and why this didn’t happen on 2.1.3?

Thanks,

Paul

2 things:

  • such crash should definitively crash the FileDataWriter. This is fixed as of https://github.com/gatling/gatling/issues/2871
  • Why RequestImpl.toString crashes with a ConcurrentModificationException is a mystery. I’m not in a rush to investigate this personally as Request implementation completely changes with AsyncHttpClient 2 and Gatling 2.2