Hi,
We are using Gatling gatling-charts-highcharts-2.0.0-M2 to performance test some of our products and we are facing issues when testing above 110 TPS. Requests start to pile up and eventually timing out (after around 10 secs instead of the 60 configured tho).
It does not happen with jmeter and we’ve tried several servers (original one was spray.io, but we have tested java netty and apache httpd just to be sure).
We have tried closing the connections adding the Connection: close header, deactivating request pooling in gatling.conf, increasing the akka dispatcher throughput to no avail
This is the test we are using against an apache httpd server
import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
import io.gatling.http.Headers.Names._
import scala.concurrent.duration._
import bootstrap._
import assertions._
class SimpleSimulation extends Simulation {
val httpConf = httpConfig
.baseURL(“http://www.techdelivery.es”)
.acceptCharsetHeader(“ISO-8859-1,utf-8;q=0.7,;q=0.7")
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,/*;q=0.8”)
//.acceptEncodingHeader(“gzip, deflate”)
.acceptLanguageHeader(“fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3”)
.disableFollowRedirect
val headers_1 = Map(
“Connection” → “close”)
val scn = scenario(“Basic request”)
.group(“Index”) {
exec(
http(“get_index”)
.get("/index.html")
.headers(headers_1)
.check(status.is(200)))
}
setUp(scn.inject(ramp(1200 users) over(10 seconds)).protocolConfig(httpConf))
Hi,
Can I hammer this same server?
Sure
It is an apache server with a static index.html. I have a spray.io server in the port 9080 , file /index.html also
OK, I think I know what happens, probably this as you’re launching tons of new users that never reuse their connection.
Could you add .shareConnections to your HttpProtocolConfig until I fix this, please?
Sure, will try that! Thanks a lot
Could I get your JMeter scenario too, please?
OK, I think this is actually not a bug, but I’d like your help to confirm.
The default behavior of a user is the one of a browser instance: it has it’s own connection pool, meaning that if you start 2000 users, you’ll get 2000 open connections. And we don’t currently close connections when a user ends because that’s not how browsers work: the connection usually stays open until it times out (usually 1 min) or the server decides to terminate them.
With your current test set up, you quickly get hundreds of open connections, and your server ends up not being able to connect any more.
I tested with JMeter w/ HttpClient 4 and I have the same behavior. Could I get your jmx file, please?
I sent a response previously, but looks like it didn’t go through. It still does not work with .shareConnections
I’m attaching the jmx file
Thanks a million, you rock
SampleRequest.jmx (4.95 KB)
In your JMX, you have 200 users ramped over 20 sec.
If I raise to 1200 users over 10 sec like in your Gatling scenario, it has the same Connection problems, just that it aborts after 60 seconds because of the Runtime Controller.
In Gatling did you remove your Connection: Close header?
Yes, I have 200 users but don’t stop sending requests. We are building REST APIs so we are interested in pure HTTP call performance, not process performance. Maybe there is a better way to test a REST API, using a smaller number of users but repeating requests for that connection with a feeder?
Yes, I tried removing the connection: close header
We are building REST APIs so we are interested in pure HTTP call
performance, not process performance. Maybe there is a better way to test a
REST API, using a smaller number of users but repeating requests for that
connection with a feeder?
If that's what you're looking for and you don't want to take the connection
factor into account, you can use a loop (beware, during was broken in
2.0.0-M2, but you can use repeat).
Here's my Simulation for 2.0.0-SNAPSHOT (a build is under way on Cloudbees):
class SimpleSimulation extends Simulation {
val httpConf = http
.baseURL("http://www.techdelivery.es")
.acceptCharsetHeader("ISO-8859-1,utf-8;q=0.7,*;q=0.7")
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8")
.acceptLanguageHeader("fr,fr-fr;q=0.8,en-us;q=0.5,en;q=0.3")
.disableFollowRedirect
.shareConnections
val scn = scenario("Basic request")
.during(60) {
exec(
http("get_index")
.get("/index.html"))
}
setUp(scn.inject(ramp(200 users) over (20 seconds))).protocols(httpConf)
}
and here are the results.
I did got connection timeouts (and that's a server problem), Gatling ran
smoothly, and max rps was ~1500.
simplesimulation-20130531172509.zip (235 KB)