I’m seeing the below a lot when running my tests having upgraded to 2.2.0. I didn’t see these errors before upgrading.
The site in question doesn’t appear to be the issue - I can’t find any common link between the sites that cause the crash.
`
13:11:06.833 [ERROR] i.g.h.a.ResponseProcessor - AsyncHandlerActor crashed on tx HttpTx(Session(No proxy Users (0),38,Map(gatling.http.cache.dns → io.gatling.http.resolver.ShuffleJdkNameResolver@8a8b440, url → milwaukeeinsurance.jobs, timestamp.833c9b5d-e846-4934-9ef2-03e11ab5f618 → 1461931866342, 0ddcc6fe-b6e5-47c2-aeb9-1825732a0b04 → 0, term → giving usa 2012 report, link → http://veterans.jobs/, 833c9b5d-e846-4934-9ef2-03e11ab5f618 → 0, timestamp.0ddcc6fe-b6e5-47c2-aeb9-1825732a0b04 → 1461931866342, gatling.http.cookies → CookieJar(Map(CookieKey(sessionid,milwaukeeinsurance.jobs,/) → StoredCookie(sessionid=317tulq2477spttntefut1auny4emaod; path=/; maxAge=1209600s; HTTPOnly,true,true,1461931866684), CookieKey(aguid,my.jobs,/) → StoredCookie(aguid=2a031ab961d140d8b9ebbfcad2b5174b; domain=.my.jobs; path=/,false,false,1461931866649)))),1461931859434,22,OK,List(ExitOnCompleteLoopBlock(833c9b5d-e846-4934-9ef2-03e11ab5f618), TryMaxBlock(0ddcc6fe-b6e5-47c2-aeb9-1825732a0b04,io.gatling.core.action.InnerTryMax@2bdc07e6,OK)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$208/1949126165@19af0ea0),HttpRequest(jqsmoothness.css,http://d2e48ltfsb5exy.cloudfront.net/framework/css/code/jqsmoothness.css GET headers: Accept:text/css,/;q=0.1 DNT:1 Accept-Language:en-US,en;q=0.5 Accept-Encoding:gzip, deflate User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36 Chrome 45.0,HttpRequestConfig(List(HttpCheck(CheckBase(io.gatling.http.check.HttpCheckBuilders$$$Lambda$41/2038105753@77a6de10,io.gatling.core.session.package$ExpressionSuccessWrapper$$$Lambda$39/1345401730@36d52863,,None),Status,None)),None,None,Some(5),false,None,true,true,CoreComponents(Actor[akka://GatlingSystem/user/gatling-controller#-930363237],io.gatling.core.controller.throttle.Throttler@2d7d4ad9,io.gatling.core.stats.DataWritersStatsEngine@32f4e925,io.gatling.core.action.Exit@126a833e,GatlingConfiguration(CoreConfiguration(2.2.0,None,None,utf-8,None,ExtractConfiguration(RegexConfiguration(200),XPathConfiguration(200),JsonPathConfiguration(200,false),CssConfiguration(200)),DirectoryConfiguration(user-files/data,user-files/bodies,user-files/simulations,None,None,results),false,200,200,1000),ChartingConfiguration(false,1000,false,IndicatorsConfiguration(800,1200,50.0,75.0,95.0,99.0)),HttpConfiguration(200,200,200,Some(http://gatling.io),true,SslConfiguration(None,None),AhcConfiguration(true,60000,60000,60000,2,60000,true,4096,8192,8192,10240000,List(TLSv1.2, TLSv1.1, TLSv1),List(),0,0,false,false,true,true,false,-1,-1,-1),DnsConfiguration(5000,3)),DataConfiguration(ArrayBuffer(io.gatling.core.stats.writer.ConsoleDataWriterType$@7c17d10e, io.gatling.core.stats.writer.FileDataWriterType$@3bfb67d7),FileDataWriterConfiguration(8192),LeakDataWriterConfiguration(30),ConsoleDataWriterConfiguration(false),GraphiteDataWriterConfiguration(false,localhost,2003,Tcp,gatling,8192,1)),Config(SimpleConfigObject({“awt”:{“toolkit”:“sun.lwawt.macosx.LWCToolkit”},“file”:{“encoding”:{“pkg”:“sun.io”},“separator”:"/"},“ftp”:{“nonProxyHosts”:“local|.local|169.254/16|.169.254/16”},“gatling”:{“charting”:{“indicators”:{“higherBound”:1200,“lowerBound”:800,“percentile1”:50,“percentile2”:75,“percentile3”:95,“percentile4”:99},“maxPlotPerSeries”:1000,“noReports”:false,“useGroupDurationMetric”:false},“core”:{“directory”:{“binaries”:"",“bodies”:“user-files/bodies”,“data”:“user-files/data”,“reportsOnly”:"",“results”:“results”,“simulations”:“user-files/simulations”},“elFileBodiesCacheMaxCapacity”:200,“encoding”:“utf-8”,“extract”:{“css”:{“cacheMaxCapacity”:200},“jsonPath”:{“cacheMaxCapacity”:200,“preferJackson”:false},“regex”:{“cacheMaxCapacity”:200},“xpath”:{“cacheMaxCapacity”:200}},“mute”:false,“outputDirectoryBaseName”:"",“rawFileBodiesCacheMaxCapacity”:200,“rawFileBodiesInMemoryMaxSize”:1000,“runDescription”:"",“simulationClass”:""},“data”:{“console”:{“light”:false},“file”:{“bufferSize”:8192},“graphite”:{“bufferSize”:8192,“host”:“localhost”,“light”:false,“port”:2003,“protocol”:“tcp”,“rootPathPrefix”:“gatling”,“writeInterval”:1},“leak”:{“noActivityTimeout”:30},“writers”:[“console”,“file”]},“http”:{“ahc”:{“acceptAnyCertificate”:true,“connectTimeout”:60000,“httpClientCodecMaxChunkSize”:8192,“httpClientCodecMaxHeaderSize”:8192,“httpClientCodecMaxInitialLineLength”:4096,“keepAlive”:true,“maxRetry”:2,“pooledConnectionIdleTimeout”:60000,“readTimeout”:60000,“requestTimeout”:60000,“soLinger”:-1,“soRcvBuf”:-1,“soReuseAddress”:false,“soSndBuf”:-1,“sslEnabledCipherSuites”:[],“sslEnabledProtocols”:[“TLSv1.2”,“TLSv1.1”,“TLSv1”],“sslSessionCacheSize”:0,“sslSessionTimeout”:0,“tcpNoDelay”:true,“useNativeTransport”:false,“useOpenSsl”:false,“usePooledMemory”:true,“webSocketMaxFrameSize”:10240000},“dns”:{“maxQueriesPerResolve”:3,“queryTimeout”:5000},“enableGA”:true,“fetchedCssCacheMaxCapacity”:200,“fetchedHtmlCacheMaxCapacity”:200,“perUserCacheMaxCapacity”:200,“ssl”:{“keyStore”:{“algorithm”:"",“file”:"",“password”:"",“type”:""},“trustStore”:{“algorithm”:"",“file”:"",“password”:"",“type”:""}},“warmUpUrl”:“http://gatling.io”}},“gopherProxySet”:“false”,“http”:{“nonProxyHosts”:“local|.local|169.254/16|.169.254/16”},“java”:{“awt”:{“graphicsenv”:“sun.awt.CGraphicsEnvironment”,“printerjob”:“sun.lwawt.macosx.CPrinterJob”},“class”:{“path”:"/Users/robertgates55/web_security_performance_gatling/lib/akka-actor_2.11-2.4.3.jar:/Users/robertgates55/web_security_performance_gatling/lib/akka-slf4j_2.11-2.4.3.jar:/Users/robertgates55/web_security_performance_gatling/lib/async-http-client-2.0.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/bcpkix-jdk15on-1.54.jar:/Users/robertgates55/web_security_performance_gatling/lib/bcprov-jdk15on-1.54.jar:/Users/robertgates55/web_security_performance_gatling/lib/boon-json-0.5.7-RC1.jar:/Users/robertgates55/web_security_performance_gatling/lib/boon-reflekt-0.5.7-RC1.jar:/Users/robertgates55/web_security_performance_gatling/lib/boopickle_2.11-1.1.3.jar:/Users/robertgates55/web_security_performance_gatling/lib/commons-pool-1.6.jar:/Users/robertgates55/web_security_performance_gatling/lib/concurrentlinkedhashmap-lru-1.4.2.jar:/Users/robertgates55/web_security_performance_gatling/lib/config-1.3.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/fastring_2.11-0.2.4.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-app-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-charts-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-charts-highcharts-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-commons-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-core-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-http-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-jdbc-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-jms-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-metrics-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-recorder-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/gatling-redis-2.2.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/geronimo-jms_1.1_spec-1.1.1.jar:/Users/robertgates55/web_security_performance_gatling/lib/HdrHistogram-2.1.8.jar:/Users/robertgates55/web_security_performance_gatling/lib/jackson-annotations-2.7.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/jackson-core-2.7.3.jar:/Users/robertgates55/web_security_performance_gatling/lib/jackson-databind-2.7.3.jar:/Users/robertgates55/web_security_performance_gatling/lib/jackson-dataformat-csv-2.7.3.jar:/Users/robertgates55/web_security_performance_gatling/lib/javassist-3.20.0-GA.jar:/Users/robertgates55/web_security_performance_gatling/lib/jodd-core-3.7.jar:/Users/robertgates55/web_security_performance_gatling/lib/jodd-lagarto-3.7.jar:/Users/robertgates55/web_security_performance_gatling/lib/jodd-log-3.7.jar:/Users/robertgates55/web_security_performance_gatling/lib/jsonpath_2.11.jar:/Users/robertgates55/web_security_performance_gatling/lib/logback-classic-1.1.7.jar:/Users/robertgates55/web_security_performance_gatling/lib/logback-core-1.1.7.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-buffer-4.0.36.Final.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-codec-4.0.36.Final.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-codec-dns-2.0.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-codec-http-4.0.36.Final.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-common-4.0.36.Final.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-handler-4.0.36.Final.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-reactive-streams-1.0.4.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-resolver-2.0.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-resolver-dns-2.0.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-transport-4.0.36.Final.jar:/Users/robertgates55/web_security_performance_gatling/lib/netty-transport-native-epoll-4.0.36.Final-linux-x86_64.jar:/Users/robertgates55/web_security_performance_gatling/lib/quicklens_2.11-1.4.6.jar:/Users/robertgates55/web_security_performance_gatling/lib/reactive-streams-1.0.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/redisclient_2.11-3.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/Saxon-HE-9.7.0-4.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-compiler-2.11.8.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-java8-compat_2.11-0.7.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-library-2.11.8.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-logging_2.11-3.1.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-parser-combinators_2.11-1.0.4.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-reflect-2.11.8.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-swing_2.11-1.0.2.jar:/Users/robertgates55/web_security_performance_gatling/lib/scala-xml_2.11-1.0.5.jar:/Users/robertgates55/web_security_performance_gatling/lib/scopt_2.11-3.4.0.jar:/Users/robertgates55/web_security_performance_gatling/lib/slf4j-api-1.7.21.jar:/Users/robertgates55/web_security_performance_gatling/lib/t-digest-3.1.jar:/Users/robertgates55/web_security_performance_gatling/user-files:/Users/robertgates55/web_security_performance_gatling/conf:",“version”:“52.0”},“endorsed”:{“dirs”:"/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/endorsed"},“ext”:{“dirs”:"/Users/robertgates55/Library/Java/Extensions:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/ext:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java"},“home”:"/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre",“io”:{“tmpdir”:"/var/folders/6z/hwppzvh540d820sm8w9b1sdh0000gp/T/"},“library”:{“path”:"/Users/robertgates55/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:."},“net”:{“preferIPv4Stack”:“true”,“preferIPv6Addresses”:“false”},“runtime”:{“name”:“Java™ SE Runtime Environment”,“version”:“1.8.0_31-b13”},“specification”:{“name”:“Java Platform API Specification”,“vendor”:“Oracle Corporation”,“version”:“1.8”},“vendor”:{“url”:{“bug”:“http://bugreport.sun.com/bugreport/"}},“version”:“1.8.0_31”,“vm”:{“info”:"mixed mode”,“name”:“Java HotSpot™ 64-Bit Server VM”,“specification”:{“name”:“Java Virtual Machine Specification”,“vendor”:“Oracle Corporation”,“version”:“1.8”},“vendor”:“Oracle Corporation”,“version”:“25.31-b07”}},“line”:{“separator”:"\n"},“os”:{“arch”:“x86_64”,“name”:“Mac OS X”,“version”:“10.10.2”},“path”:{“separator”:":"},“socksNonProxyHosts”:“local|.local|169.254/16|.169.254/16”,“sun”:{“arch”:{“data”:{“model”:“64”}},“boot”:{“class”:{“path”:"/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/sunrsasign.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/classes"},“library”:{“path”:"/Library/Java/JavaVirtualMachines/jdk1.8.0_31.jdk/Contents/Home/jre/lib"}},“cpu”:{“endian”:“little”,“isalist”:""},“io”:{“unicode”:{“encoding”:“UnicodeBig”}},“java”:{“command”:“io.gatling.app.Gatling”,“launcher”:“SUN_STANDARD”},“jnu”:{“encoding”:“UTF-8”},“management”:{“compiler”:“HotSpot 64-Bit Tiered Compilers”},“os”:{“patch”:{“level”:“unknown”}}},“user”:{“country”:{“format”:“GB”},“dir”:"/Users/robertgates55/web_security_performance_gatling",“home”:"/Users/robertgates55",“language”:“en”,“name”:“robertgates55”,“timezone”:“Europe/London”}})))),HttpComponents(HttpProtocol(List(),Some(http://gatling.io),HttpProtocolEnginePart(true,false,6,false,Map(),None,List()),HttpProtocolRequestPart(Map(Accept → , DNT → , Accept-Language → , Accept-Encoding → , User-Agent → ),None,false,false,false,None,true,None),HttpProtocolResponsePart(true,Some(5),false,true,None,List(),None,true,None),HttpProtocolWsPart(List(),false,None),HttpProtocolProxyPart(None,List())),io.gatling.http.ahc.HttpEngine@7c997144,io.gatling.http.cache.HttpCaches@24e997dd,io.gatling.http.ahc.ResponseProcessor@577211c2),List())),io.gatling.http.response.ResponseBuilder$$$Lambda$194/1907467420@2b52ef02,io.gatling.core.action.Pause@7f73a495,Some(Actor[akka://GatlingSystem/user/resourceFetcher-130#299735574]),0,io.gatling.core.session.Session$$$Lambda$226/2128742070@3365dd45), forwarding user to the next action
java.nio.charset.CharacterCodingException: null
at io.gatling.http.util.Utf8ByteBuffersDecoder.write(ByteBuffersDecoder.scala:135)
at io.gatling.http.util.Utf8ByteBuffersDecoder.io$gatling$http$util$Utf8ByteBuffersDecoder$$$anonfun$6(ByteBuffersDecoder.scala:144)
at io.gatling.http.util.Utf8ByteBuffersDecoder.io$gatling$http$util$Utf8ByteBuffersDecoder$$$anonfun$6$adapted(ByteBuffersDecoder.scala:142)
at io.gatling.http.util.Utf8ByteBuffersDecoder$$Lambda$311/442344649.apply(Unknown Source)
at scala.collection.immutable.List.foreach(List.scala:381)
at io.gatling.http.util.Utf8ByteBuffersDecoder.decode(ByteBuffersDecoder.scala:142)
at io.gatling.http.util.ByteBuffersDecoder$.decode(ByteBuffersDecoder.scala:52)
at io.gatling.http.util.BytesHelper$.byteArraysToString(BytesHelper.scala:61)
at io.gatling.http.response.InputStreamResponseBody.liftedTree1$1(ResponseBody.scala:125)
at io.gatling.http.response.InputStreamResponseBody.string$lzycompute(ResponseBody.scala:124)
at io.gatling.http.response.InputStreamResponseBody.string(ResponseBody.scala:120)
at io.gatling.http.ahc.ResponseProcessor.executeNext(ResponseProcessor.scala:152)
at io.gatling.http.ahc.ResponseProcessor.logAndExecuteNext(ResponseProcessor.scala:169)
at io.gatling.http.ahc.ResponseProcessor.checkAndProceed$1(ResponseProcessor.scala:291)
at io.gatling.http.ahc.ResponseProcessor.processResponse(ResponseProcessor.scala:321)
at io.gatling.http.ahc.ResponseProcessor.onCompleted(ResponseProcessor.scala:61)
at io.gatling.http.ahc.AsyncHandler.onCompleted(AsyncHandler.scala:127)
at io.gatling.http.ahc.AsyncHandler.onCompleted(AsyncHandler.scala:46)
at org.asynchttpclient.netty.NettyResponseFuture.getContent(NettyResponseFuture.java:188)
at org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:223)
at org.asynchttpclient.netty.handler.HttpHandler.finishUpdate(HttpHandler.java:58)
at org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:159)
at org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:189)
at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:82)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:428)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:277)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:264)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:243)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:292)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:278)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:962)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:485)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:399)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:371)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
`
I can’t seem to replicate it directly (eg with a single call to the site in the above example), but it seems to happen when running in parallel with other requests, and happens more often at higher requests/sec rates.
I believe this (or a related issue) is currently causing gatling to make far fewer requests than it should do - the requests/sec rates I’m seeing having upgraded are far lower.
Thanks,
Rob
