[Gatling 2.0.0-M3] Actor AsyncHandlerActor doesn't support message ReceiveTimeout

Excilys · July 10, 2013, 2:43pm

I guess I’ll have no choice but to check all of our Actors to see if one doesn’t implement exception handling logic…
I hope the message flow is not broken somewhere in AHC/Netty…

Stefan_Magnus_Landro · July 10, 2013, 6:08pm

Should I try to reproduce with grizzly?

Excilys · July 10, 2013, 6:50pm

That’s a good idea!

Frode_Nerbraten · July 11, 2013, 5:34am

@Frode: regarding your BigIp problem, did you check this with something like Charles or Wireshark? I’d hate this to be a kind of buffer overflow in Gatling/AHC/Netty…

The symptom I’m seeing is actually the same as in this issue: https://issues.jboss.org/browse/NETTY-369
But it looks like I get two responses from the same request. This makes the next request fail before it is even sent to the server:

06:35:31.643 [INFO ] i.g.h.a.HttpRequestAction - Sending request ‘upload content’: scenario ‘Scenario name’, userId #0
06:35:32.015 [INFO ] i.g.h.a.HttpRequestAction - Sending request ‘add receipients’: scenario ‘Scenario name’, userId #0
06:35:32.039 [WARN ] i.g.h.a.AsyncHandler - Request ‘add receipients’ failed
java.lang.IllegalArgumentException: invalid version format: <!DOCTYPE at org.jboss.netty.handler.codec.http.HttpVersion.(HttpVersion.java:102) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpVersion.valueOf(HttpVersion.java:62) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpResponseDecoder.createMessage(HttpResponseDecoder.java:104) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:189) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:143) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:127) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:435) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.6.6.Final.jar:na]

In Charles the ‘upload content’ multipart request is marked as OK, and then there is an empty failed request after it without any path, http verb or anything. So I’m pretty sure the ‘add recipients’ request is never sent. The timing is important, because if I add 2 seconds sleep after the upload request, the subsequent requests succeeds. But I still get an extra empty failed request in Charles - one that I never send and I suspect is the extra response body coming from BigIp. In this case I get an exception from a different place (the same NPE as Stefan), but it’s the same cause. The stacktrace:

07:15:22.563 [INFO ] i.g.c.a.Pause - Pausing for 2000ms (real=1764ms)
07:15:22.564 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Unexpected I/O exception on channel [id: 0x078f0607, /127.0.0.1:49803 => localhost/127.0.0.1:8888]
java.lang.NullPointerException: null
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.isContentAlwaysEmpty(HttpClientCodec.java:179) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readHeaders(HttpMessageDecoder.java:514) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:193) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:143) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:127) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:485) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.6.6.Final.jar:na]

Using the debugger I can see that when the exception is thrown from org.jboss.netty.handler.codec.http.HttpVersion.java:102 the channelbuffer in HttpMessageDecoder contains the html document below and nothing else - explaining the exception since it is expecting som http headers first:

400 Bad Request

Bad Request

Your browser sent a request that this server could not understand.

Im 98% sure this response does not come from our app-server because I have stepped through every line of jersey/jetty hendler logic and cannot find anything So I’m suspecting BigIp is doing something strange.
But It doesn’t look like Netty mixes up the requests/responses I’m sending, only that it gets one additional unexpected response. The ‘upload content’ response has a correct content-length and I can see that the response body and headers are correct when sent from the server and received by gatling (or Charles).

I hope this makes some sense to you The only thing that works reliably for me is to set Connection: close in every request. Then everything works.

Regards,
Frode

Excilys · July 11, 2013, 6:29am

Just to be sure: could you try the Grizzly provider instead of the Netty one, please?

Frode_Nerbraten · July 11, 2013, 6:34am

Just to be sure: could you try the Grizzly provider instead of the Netty one, please?

Just tried it, but ran into another problem. The grizzly provider sets Content-Encoding:gzip in every request, even GET requests. This causes an exception on the server in Jersey’s GZIPContentEncodingFilter How can I prevent this?

GET /post/api HTTP/1.1
Content-Encoding: gzip
…
Accept-Encoding: gzip

Frode

Frode_Nerbraten · July 11, 2013, 6:50am

Ok, tried to set ahc.compressionEnabled = false and got rid of the Content-Encoding, but it still fails when reading the response from a simple GET (even though the response code is OK and I can see the correct json in the log):

08:38:58.108 [WARN ] i.g.h.a.AsyncHandler - Request ‘get root’ failed
java.io.IOException: Remotely Closed
at com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientTransportFilter$1.failed(GrizzlyAsyncHttpProvider.java:749) [async-http-client-1.7.19.20130706.jar:na]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.notifyFailure(DefaultFilterChain.java:606) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.throwChain(DefaultFilterChain.java:490) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:164) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:837) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55) [grizzly-framework-2.3.3.jar:2.3.3]
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135) [grizzly-framework-2.3.3.jar:2.3.3]

Request:
get root: KO Remotely Closed

Excilys · July 11, 2013, 7:51am

Grizzly support just made it back into Gatling. We don’t actually do anything special, just enable it in AHC.
I honestly haven’t tested it on complex scenarios yet.

What does your Gatling multipart code look like? This part is a bit messy in AHC and the implementation depends on the protocol and the type of multipart.

Frode_Nerbraten · July 11, 2013, 8:09am

My multipart code looks like this: (I tried the ByteArrayBodyPart also, but no difference)

http(“upload content”)
.post(_(“message”).as[Message].link(“add_content”))
.bodyPart(StringBodyPart(“token”, “${csrfToken}”))
.bodyPart(RawFileBodyPart(“file”, filename, “application/octet-stream”).withFileName(filename))
.header(“Accept”, “/”)
.check(status.is(200), bodyString.is(“OK”))

This request works fine on our server and the response is ok and the check succeeds. After that I get an exception because of the BAD REQUEST html document. It’s a bit tricky to see the exact raw data being sent over the connection because it’s SSL. I cant use Wireshark because I don’t have the private key from bigip.

Excilys · July 11, 2013, 8:20am

Do you also experience this problem with other user-agents (browsers or others, I don’t know your use case)?

I’m afraid we won’t be able do investigate this any further before I refactor the multipart support in AHC and build proper test cases on my own. I won’t probably be able to work on this before september.

Frode_Nerbraten · July 11, 2013, 8:50am

Do you also experience this problem with other user-agents (browsers or others, I don’t know your use case)?

It’s a rest api used by different types of clients (javascript, ios, android) and I haven’t encountered this with any of them. Although I had som problems using Grinder and Java 7 that caused requests to hang. It could be related - I don’t know.

I’m afraid we won’t be able do investigate this any further before I refactor the multipart support in AHC and build proper test cases on my own. I won’t probably be able to work on this before september.

I totally understand that, and I’m still suspecting that there is something wrong with our bigip config. I might do some more digging, but for now my simulations work using connection close. I can live with that

Thank you for looking into it and fixing those other issues so quickly!

Frode

Stefan_Magnus_Landro · July 11, 2013, 10:44am

I added some debug logging to the netty provider. There might be some clues in there (but I’m afraid I’m clueless…):

11:58:20.305 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Using cached Channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
for uri https://myhost/service

11:58:20.306 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider -
Using cached Channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
for request
DefaultHttpRequest(chunked: false)
POST /service HTTP/1.1
Host: myhost
Accept-Encoding: gzip
Connection: keep-alive
Accept: /
User-Agent: Ytelsestest
Content-Type: multipart/related; type=“application/xop+xml”; start=“root.message@cxf.apache.org”; start-info=“text/xml”; boundary=ERxmkV2ue2hB9ZL1hjEmtcl1gaNcv40UVdmr
Content-Length: 715907

11:58:20.420 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider -

Request DefaultHttpRequest(chunked: false)
POST /service HTTP/1.1
Host: myhost
Accept-Encoding: gzip
Connection: keep-alive
Accept: /
User-Agent: Ytelsestest
Content-Type: multipart/related; type=“application/xop+xml”; start=“root.message@cxf.apache.org”; start-info=“text/xml”; boundary=ERxmkV2ue2hB9ZL1hjEmtcl1gaNcv40UVdmr
Content-Length: 715907

Response DefaultHttpResponse(chunked: false)
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: multipart/related; type=“application/xop+xml”; boundary=“uuid:7f8f6918-1ae3-4b16-9c44-96969ed1df2e”; start=“root.message@cxf.apache.org”; start-info=“text/xml”
Content-Length: 690

11:58:20.420 [DEBUG] c.n.h.c.p.n.NettyConnectionsPool - Adding uri: https://myhost for channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
11:58:20.421 [INFO ] i.g.c.a.Pause - Pausing for 5000ms (real=4988ms)
11:58:20.427 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Unexpected I/O exception on channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method) ~[na:1.7.0_04]
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) ~[na:1.7.0_04]
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218) ~[na:1.7.0_04]
at sun.nio.ch.IOUtil.read(IOUtil.java:186) ~[na:1.7.0_04]
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359) ~[na:1.7.0_04]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.6.6.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_04]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_04]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_04]
11:58:20.427 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Closing Channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
11:58:20.432 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Unexpected I/O exception on channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
java.lang.NullPointerException: null
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.isContentAlwaysEmpty(HttpClientCodec.java:179) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readHeaders(HttpMessageDecoder.java:514) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:193) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:143) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.decode(HttpClientCodec.java:127) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:485) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.6.6.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_04]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_04]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_04]
11:58:20.432 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Closing Channel [id: 0xf4318226, /10.33.56.173:34443 => myhost/10.33.56.110:8443]
11:58:20.433 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - null
java.nio.channels.ClosedChannelException: null
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:406) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:374) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:58) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:779) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.closeOutboundAndChannel(SslHandler.java:1457) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.handleDownstream(SslHandler.java:590) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleDownstream(HttpClientCodec.java:97) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendDownstream(DefaultChannelPipeline.java:784) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:109) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:591) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendDownstream(DefaultChannelPipeline.java:582) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.close(Channels.java:812) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:197) [netty-3.6.6.Final.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.finishChannel(NettyAsyncHttpProvider.java:1107) [async-http-client-1.7.19.20130706.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.closeChannel(NettyAsyncHttpProvider.java:1093) [async-http-client-1.7.19.20130706.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.exceptionCaught(NettyAsyncHttpProvider.java:1551) [async-http-client-1.7.19.20130706.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.exceptionCaught(SimpleChannelUpstreamHandler.java:153) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.exceptionCaught(SslHandler.java:695) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:112) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.notifyHandlerException(DefaultChannelPipeline.java:658) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:566) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) [netty-3.6.6.Final.jar:na]
at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42) [netty-3.6.6.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_04]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_04]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_04]
11:58:20.434 [DEBUG] o.j.n.h.s.SslHandler - Swallowing an exception raised while writing non-app data
java.nio.channels.ClosedChannelException: null
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.cleanUpWriteBuffer(AbstractNioWorker.java:406) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:374) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:58) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.closeOutboundAndChannel(SslHandler.java:1457) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.handleDownstream(SslHandler.java:590) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleDownstream(HttpClientCodec.java:97) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.close(Channels.java:812) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:197) ~[netty-3.6.6.Final.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.finishChannel(NettyAsyncHttpProvider.java:1107) ~[async-http-client-1.7.19.20130706.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.closeChannel(NettyAsyncHttpProvider.java:1093) ~[async-http-client-1.7.19.20130706.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.exceptionCaught(NettyAsyncHttpProvider.java:1551) ~[async-http-client-1.7.19.20130706.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.exceptionCaught(SslHandler.java:695) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.6.6.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_04]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_04]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_04]
11:58:20.435 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Unexpected I/O exception on channel [id: 0xf4318226, /10.33.56.173:34443 :> myhost/10.33.56.110:8443]
java.nio.channels.ClosedChannelException: null
at org.jboss.netty.handler.stream.ChunkedWriteHandler.discard(ChunkedWriteHandler.java:168) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.flush(ChunkedWriteHandler.java:192) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:137) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpContentDecoder.channelClosed(HttpContentDecoder.java:147) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.replay.ReplayingDecoder.cleanup(ReplayingDecoder.java:570) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.channelClosed(FrameDecoder.java:371) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec$Decoder.channelClosed(HttpClientCodec.java:221) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.cleanup(FrameDecoder.java:493) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.channelClosed(FrameDecoder.java:371) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.channelClosed(SslHandler.java:1605) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireChannelClosed(Channels.java:468) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.close(AbstractNioWorker.java:376) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:58) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.closeOutboundAndChannel(SslHandler.java:1457) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.handleDownstream(SslHandler.java:590) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.oneone.OneToOneEncoder.handleDownstream(OneToOneEncoder.java:54) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleDownstream(HttpClientCodec.java:97) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleDownstream(ChunkedWriteHandler.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.close(Channels.java:812) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:197) ~[netty-3.6.6.Final.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.finishChannel(NettyAsyncHttpProvider.java:1107) ~[async-http-client-1.7.19.20130706.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.closeChannel(NettyAsyncHttpProvider.java:1093) ~[async-http-client-1.7.19.20130706.jar:na]
at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.exceptionCaught(NettyAsyncHttpProvider.java:1551) ~[async-http-client-1.7.19.20130706.jar:na]
at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:142) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.exceptionCaught(FrameDecoder.java:377) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:92) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.ssl.SslHandler.exceptionCaught(SslHandler.java:695) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireExceptionCaught(Channels.java:525) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:48) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90) ~[netty-3.6.6.Final.jar:na]
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178) ~[netty-3.6.6.Final.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [na:1.7.0_04]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [na:1.7.0_04]
at java.lang.Thread.run(Thread.java:722) [na:1.7.0_04]
11:58:20.439 [DEBUG] c.n.h.c.p.n.NettyAsyncHttpProvider - Channel Closed: [id: 0xf4318226, /10.33.56.173:34443 :> myhost/10.33.56.110:8443] with attachment com.ning.http.client.providers.netty.NettyAsyncHttpProvider$DiscardEvent@175e895d

Stefan_Magnus_Landro · August 12, 2013, 10:16am

Hi there,

Just got back from vacation and re-ran our tests with 2.0.0-SNAPSHOT containing the AHC 1.7.19 fixes. Now, everything works as expected - thanks Stephane!

Stefan

Excilys · August 27, 2013, 8:01am

Glad to hear!

Which configuration did you try? Send a file part or a byte array? Those are not handled the same way, and I was hoping both would have been fixed.

Cheers,

Stéphane

Stefan_Magnus_Landro · August 27, 2013, 2:33pm

RawFilePart is what i use here.

Cheers

Stefan

Excilys · August 27, 2013, 2:38pm

Nice, that’s the more efficient one.

So damn glad to be done with this multipart stuff (until rewriting it for AHC2…)!
Thanks a lot for your help and patience with this one…

Stefan_Magnus_Landro · August 27, 2013, 3:08pm

Thanks to you too. You can’t imagine how important this is to us.

Cheers

Stefan

Topic		Replies	Views
Test hangs when there are lots of failures Gatling (Open-Source)	19	262	February 14, 2013
Issues with HTTP-Body Gatling (Open-Source)	4	198	May 1, 2016
Issue with multipart form data upload in Gatling 2.0.0-M3 Gatling (Open-Source)	26	1200	January 9, 2014
Gatling Multipart Requests over SSL Gatling (Open-Source)	13	183	August 8, 2014
[GATLING 2] NPE when downloading mulitpart response Gatling (Open-Source)	0	148	June 10, 2013

[Gatling 2.0.0-M3] Actor AsyncHandlerActor doesn't support message ReceiveTimeout

Bad Request

Related topics