Performance numbers mismatch between gatling and ab (apache benchmark)

Gatling (Open-Source)
1

Hi Gatling Users,

We have written a service framework which we wanted to test. We have used both gatling and ab to test the service written on this framework.
Our test is a simple HTTP Get on a jersey resource which returns a string (application/text) back.

I know gatling is normally used for more complex scenarios but i was not expecting so much mismatch between throughput of gatling and ab for a scenario of

gatling : Users 300 , Request/user 10,000 => throughput ~ 11.5K
ab : Requests 3000000 , concurrency 300 => throughput ~ 19.1 K

I think gatling is unable to overwhelm the server with requests that is the reason the throughput seems less in case of gatling.

Is it the case that gatling should not be used in scenario of getting raw throughput of server for a single api ? Or am i doing something wrong here ?

T&R
H S

2

Gatling version? JDK version? gatling simulation? ab command line?

3

Also note that if you’re running on Windows, Gatling is not optimized as it’s implemented on top of epoll (because that’s what Netty uses) and epoll is bogus on Windows…

4

Hi,

Thanks for quick response.

Tested on MacOs/Ubuntu both similar results. ( I am not using Epoll transport support , just using plain NIO with selector in netty)

Gatling version 2.1.1 , java version “1.7.0_75”

ab command line : ab -k -n 3000000 -c300 http://localhost:4000/plaintext

gatling scenario :

val httpConf = http
.baseURL(" http://localhost:4000") // Here is the root for all relative URLs

val text_scn = scenario(“Plain Text scenario”).repeat(10,000) {
exec(
http(“Plaintext”)
.get("/plaintext")
.check(status.is(200))
)
}

setUp(
text_scn.inject(rampUsers(300) over (15))
).protocols(httpConf)

T&R
H S

5

Tested on MacOs/Ubuntu both similar results. ( I am not using Epoll
transport support , just using plain NIO with selector in netty)

Java NIO is implemented on top of epoll system call. I was not talking
about native epoll transport from Netty 4 (Gatling still uses Netty 3).

Gatling version 2.1.1 , java version "1.7.0_75"

Could you upgrade to latest, or even better current 2.2.0-SNAPSHOT
(available from Sonatype), please?
https://oss.sonatype.org/content/repositories/snapshots/io/gatling/highcharts/gatling-charts-highcharts-bundle/2.2.0-SNAPSHOT/

gatling scenario :
val httpConf = http
    .baseURL(" http://localhost:4000") // Here is the root for all
relative URLs

Could you add .shareConnections here please, so you get one single pool
like ab, not one per virtual user? Not sure if it would change much, but
who knows.

Then, could you increase throughput in application.conf?

Finally, Gatling comes with tons of features and requests are built
dynamically, so of course all this comes with a price. Gatling wasn't means
to compete with url bashers such as ab and wrk that just send static
requests.

6

Thanks Stéphane,

I will try these and let you know if something changes.

T&R
H S