Hi,
I’ve spent some time to figure out how to configure the Http Client to use chunked encoding. I’ve tried setting the header Transfer-Encoding: chunked
My code is
val scn = scenario(“My scenario”)
.exec(http(“Upload”)
.post(“http://192.168.212.71:9080/file”)
.header(“Transfer-Encoding”, “chunked”)
.fileBody(“payload.bin”)
)
But the requests are not chunked.
Any way to set the request to use chunked encoding?
Thanks!
Javier
Hi,
Strange, it should work.
What’s your file’s size? AHC chunks size is hardcoded with a 8ko value.
Stéphane
Hi Stéphane,
the file is 50 MB,I’ve also tried with 30MB with the same results. If I use curl I get a correct chunked encoding transfer:
javier@gatling:~/gatling-charts-highcharts-1.4.4$ curl -v -X POST -H “Transfer-Encoding: chunked” -i --data-binary @user-files/request-bodies/payload.bin http://192.168.212.71:9080/file
POST /file HTTP/1.1
User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
Host: 192.168.212.71:9080
Accept: /
Transfer-Encoding: chunked
Content-Type: application/x-www-form-urlencoded
Expect: 100-continue
< HTTP/1.1 100 Continue
HTTP/1.1 100 Continue
< HTTP/1.1 201 Created
HTTP/1.1 201 Created
< Server: spray-can/1.1-M7
Server: spray-can/1.1-M7
< Date: Tue, 19 Mar 2013 20:01:25 GMT
< Content-Length: 0
Content-Length: 0
<
Thanks!
Javier
Hi Javier,
Would you by any chance a sample to share so I could reproduce, please?
This would save me a lot of time…
Stéphane
Hi
Do you mean the scenario? Because it is in the first post. Adding it again, it is very simple. The file is generated randomly with dd (dd if=/dev/urandom of=payload.bin bs=50M count=1)
The server is in github (also very simple spray can server):
https://github.com/javierarrieta/spray-spikes/tree/master/examples/http-server-example
The scenario:
import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import com.excilys.ebi.gatling.jdbc.Predef._
import akka.util.duration._
import bootstrap._
import assertions._
class FileUploadSimulation extends Simulation {
// your code starts here
val scn = scenario(“My scenario”)
.exec(http(“Upload”)
.post(“http://192.168.212.71:9080/file”)
.header(“Transfer-Encoding”, “chunked”)
.fileBody(“payload.bin”)
)
setUp(scn.users(18))
// your code ends here
}
Thanks for your help!
Perfect! Will try tomorrow.
Thanks!
OK, got it.
First, chunked transfer encoding is to be used when the Content-Length is unknown. This is available in AHC when providing a body in the form of an InputStream. We don’t currently expose it in Gatling as we haven’t seen a use case for it. In the case of a file, the Content-Length is perfectly known and AHC will perform a zero-copy and directly write on the Socket from the file.
Then, I think your spray-can sample uses the default BufferingRequestStreamActor that buffers the chunks before handling a full HttpRequest to your HttpServiceActor, so you’ll never match chunk events there. See https://github.com/spray/spray-can#receiving-chunked-requests
Cheers,
Stéphane
Hi
Thanks for the info. So for this to work I should modify gatling to accept an InputStream.
The spray-can is working for chunked-encoded requests, it is not aggregating them but creating the correct messages (ChunkedRequestStart, MessageChunk and ChunkedMessageEnd) because of the configuration:
request-chunk-aggregation-limit = 0
I wish I had more time to get a bit deeper in gatling internals. I will try to explore the options for the inputStream
Thanks Stéphane,
Javier
My bad, so that might not be the problem.
I will dig deeper as soon as I find time…
Ah, I realized that src/main/resources wasn’t in my classpath (don’t ask me why…).
Will check again ASAP.
I’ve introduced an inputStreamBody in Gatling 2/master, and I confirm that the body gets indeed chunked and that Spray actor gets the Chunk messages.
Stéphane
Thanks!
I will try it as soon as I can and post the results
Beware that at this point, the feature is only for Gatling 2, and there’s quite a few things that have changed and are not documented yet…