[Gatling 2] Crash

Gatling (Open-Source)
1

I’m seeing the following errors when overloading my system. What could be going on?

10:49:33.548 [ERROR] i.g.h.a.AsyncHandlerActor - AsyncHandlerActor crashed on message None, forwarding user to the next action¬
613 java.lang.UnsupportedOperationException: Response not built¬
614 ▸ at io.gatling.http.response.HttpResponse$$anonfun$uri$2.apply(Response.scala:83) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
615 ▸ at io.gatling.http.response.HttpResponse$$anonfun$uri$2.apply(Response.scala:83) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
616 ▸ at scala.Option.getOrElse(Option.scala:120) ~[scala-library-2.10.3.jar:na]¬
617 ▸ at io.gatling.http.response.HttpResponse.uri(Response.scala:83) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
618 ▸ at io.gatling.http.ahc.AsyncHandlerActor$$anonfun$updateCookies$1.apply(AsyncHandlerActor.scala:61) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
619 ▸ at io.gatling.http.ahc.AsyncHandlerActor$$anonfun$updateCookies$1.apply(AsyncHandlerActor.scala:61) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
620 ▸ at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:55) ~[scala-library-2.10.3.jar:na]¬
621 ▸ at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:55) ~[scala-library-2.10.3.jar:na]¬
622 ▸ at io.gatling.http.ahc.AsyncHandlerActor.logAndExecuteNext(AsyncHandlerActor.scala:169) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
623 ▸ at io.gatling.http.ahc.AsyncHandlerActor.io$gatling$http$ahc$AsyncHandlerActor$$ko(AsyncHandlerActor.scala:180) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
624 ▸ at io.gatling.http.ahc.AsyncHandlerActor.checkAndProceed$1(AsyncHandlerActor.scala:227) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
625 ▸ at io.gatling.http.ahc.AsyncHandlerActor.io$gatling$http$ahc$AsyncHandlerActor$$processResponse(AsyncHandlerActor.scala:242) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
626 ▸ at io.gatling.http.ahc.AsyncHandlerActor$$anonfun$receive$1.applyOrElse(AsyncHandlerActor.scala:77) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
627 ▸ at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) ~[akka-actor_2.10-2.2.3.jar:2.2.3]¬
628 ▸ at akka.actor.ActorCell.invoke(ActorCell.scala:456) ~[akka-actor_2.10-2.2.3.jar:2.2.3]¬
629 ▸ at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) ~[akka-actor_2.10-2.2.3.jar:2.2.3]¬
630 ▸ at akka.dispatch.Mailbox.run(Mailbox.scala:219) ~[akka-actor_2.10-2.2.3.jar:2.2.3]¬
631 ▸ at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) [akka-actor_2.10-2.2.3.jar:2.2.3]¬
632 ▸ at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) [scala-library-2.10.3.jar:na]¬
633 ▸ at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) [scala-library-2.10.3.jar:na]¬
634 ▸ at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) [scala-library-2.10.3.jar:na]¬
635 ▸ at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) [scala-library-2.10.3.jar:na]¬

10:49:40.587 [ERROR] i.g.h.a.AsyncHandlerActor - AsyncHandlerActor crashed on message Some(OnCompleted(HttpTx(Session(Apache simulation,5698774365956883604-5270,Map(
…
…

), forwarding user to the next action

java.lang.UnsupportedOperationException: Response not built¬
2821 ▸ at io.gatling.http.response.HttpResponse$$anonfun$uri$2.apply(Response.scala:83) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
2822 ▸ at io.gatling.http.response.HttpResponse$$anonfun$uri$2.apply(Response.scala:83) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
2823 ▸ at scala.Option.getOrElse(Option.scala:120) ~[scala-library-2.10.3.jar:na]¬
2824 ▸ at io.gatling.http.response.HttpResponse.uri(Response.scala:83) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
2825 ▸ at io.gatling.http.ahc.AsyncHandlerActor$$anonfun$updateCookies$1.apply(AsyncHandlerActor.scala:61) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
2826 ▸ at io.gatling.http.ahc.AsyncHandlerActor$$anonfun$updateCookies$1.apply(AsyncHandlerActor.scala:61) ~[gatling-http-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]¬
2827 ▸ at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:55) ~[scala-library-2.10.3.jar:na]¬
2828 ▸ at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:55) ~[scala-library-2.10.3.jar:na]¬

2

Hi Stefan,

I don’t really get how this could happen.
I just pushed some changes in order to investigate. Could you give them a try, please?

3

Hi Stéphane,

I’ve been trying to reproduce with your fix, but I can’t.

BTW, I also had issues with the simulation not terminating (a few users would keep running for ever), but can’t reproduce that either.
I’ll let you know when this happens again - it could be these two bugs are related in some way.

I noticed you upgraded AHC to the 1.8 series recently - could it be related to that?
We’re hitting a HTTPS endpoint btw, and if I remember, there were some issues in AHC that you fixed this summer - have they been added to 1.8 too?

Cheers,

Stefan

4

Hi Stéphane,

I've been trying to reproduce with your fix, but I can't.

I think there was several issues:

   - AsyncHandlerActor getting a None message: that was probably the
   timeout not being properly set up => fixed
   - AsyncHandlerActor getting a successful yet empty response: no clue at
   this point. I just made Gatling not crash but still log an error in
   logback. Did you see some errors in the logs?

BTW, I also had issues with the simulation not terminating (a few users
would keep running for ever), but can't reproduce that either.

Were those issues before or after upgrading to latest snapshot. There was a
race condition in AHC where Handler wasn't sometimes notified. I fixed this
in 1.8.1.

I'll let you know when this happens again - it could be these two bugs are
related in some way.

I noticed you upgraded AHC to the 1.8 series recently - could it be
related to that?

I fixed A LOT of issues in AHC 1.8:

Like:

   - timeouts are now properly handled => new implementation caused a
   breaking API change, hence the 1.8 upgrade
   - much much much less "Remotely closed exceptions". Most of those were
   caused by the fact that the connection was first fetched from the pool,
   then the request built, then the request written. Thus, the time window
   between connection fetch and write was quite wide and the probability of
   the server closing the connection in the meantime was quite big. The only
   situation where this makes sense is when using an SSL proxy: you might have
   to issue a CONNECT instead of the regular request, so you have to know if
   you were able to fetch from the pool or if you have to open a new one.

We're hitting a HTTPS endpoint btw, and if I remember, there were some
issues in AHC that you fixed this summer - have they been added to 1.8 too?

See milestone content and description above.
The only SSL issues left I'm aware of are security related (using a proper
default SSLContext instead of a lax one, and having a proper
HostNameVerifier), but, if they were to get fixed, I'd probably disable
them by default.