Gatling Vs Simple Curl Based Script.

Gatling (Open-Source)
1

Hello,

I am doing performance benchmarking of an application service, and I notice Gatling performance is very poor compared to simple shell script with curl command.

Following is the load profile.

I have 30 concurrent users doing a HTTP POST with a large json body read from a circular/random feeder.

  1. GATLING Scenario.

…

val nbUsers = Integer.getInteger(“users”, 1)

val myRamp = java.lang.Long.getLong(“ramp”, 0L)

val myIter = Integer.getInteger(“iter”, 1)

val myUrl = System.getProperty(“url”)

val pgReqFeeder = ssv(“requestBody.ssv”).circular

val httpConf = httpConfig

.baseURL(myUrl)

.acceptHeader("/")

val scn = scenario(“Performance Testing”)

.repeat(myIter) {

feed(RequestFile)

.exec(

http(“POST”)

.post("/something.do")

.header(“Content-Type”, “application/json”)

.body("${REQUESTBODY}")

.asJSON

.check(status.is(200)))

}

…

To Execute:

JAVA_OPTS="-Dusers=30 -Diter=100 -Durl=http://:9000 -Xms4096m -Xmx4096m" ./bin/gatling.sh -s basic.Perfloadfeeder -nr

  1. CURL BASED SCRIPT: I have a simple shell script making a curl call in a loop.

#!/usr/bin

./loadtest.sh

random.sh returns a random line from a file (HTTP POST Payload)

for i in {1…100 }
do

./random.sh | \

curl -i \

-k \

-s -w “%{http_code} %{http_connect} %{time_namelookup} %{time_connect} %{time_appconnect} %{time_pretransfer} %{time_starttransfer} %{num_connects} %{speed_upload} %{speed_download} %{time_total}\n” \

-o /dev/null \

-H ‘Accept: /’ \

-H “Content-Type: application/json” \

“http://:9000/something.do” \

-d @-

done

I have a wrapper shell script that just calls this curl command in a script 30 times and
Something like
#!/bin/sh
…
./loadtest.sh &
…
…
./loadtest.sh &

Here are the comparison numbers.

Tool: Curl Script GATLING GATLING

#Concurrent Users: 30 30 1

#Iterations 100 100 10

Start Time: 1404644781 1404648014

End Time: 1404644867 1404648454

Total Requests: 3000 3000 10

MIN(s): 0.129 0.08 0.240

MAX(s): 2.390 7.75 0.380

MEAN(s): 0.735419 4.204 0.318

STD DEV: 0.319655 2.221 0.048

Avg. TPS: 34 7 3

Both tests run on from the same m/c.
H/W: AMD 4 Core Cpu (2301 MHz each), 8G RAM.

ulimit -n → 524288

I don’t think the test needs too many ports (Only 30 concurrent users).

I would appreciate your help in figuring why i have such poor performance with gatling.

Thanks,

Vikram

2

Hi,

I would need a reproducer.

Regards,

Stéphane

3

Stephane,

Thanks for the quick response.I am attaching the files for your reference.

Gatling scenario uses feeder to read from file pgRequest.ssv (unfortunately i wont be able to supply the data). Both files have the same JSON body data with the difference in escaping the quotes and header for ssv file. The format is as follows.

head -2 user-files/data/pgRequest.ssv (Total Records: 96, Total No of chars in file: 1022406, Approx size of 1 request body = 8K)

REQUESTBODY;NONE

“{”“Field”":"“Value”", ““Field1"”:”“Value”", ““Field3"”: {”“F4"”: "“V4"”, "“F5"”: ““V5"” …} …}”;

head -1 pgRequest.txt # – 1 JSON BODY per line (Total Records: 95, Total No of chars in file: 923781C, Approx. size of 1 request body = 8K)

{“Field”:“Value”, “Field1”:“Value”, “Field3”: {“F4”: “V4”, “F5”: “V5” …} …}

Let me know your findings.

Thanks and Regards,
Vikram

gatling-curl.zip (1.7 KB)

4

BTW

Gatling ver: gatling-charts-highcharts-1.5.5-bundle.tar.gz

-Vikram

5

I don’t see anything wrong with your simulation…

Could you try upgrading to latest snapshot, please?
https://github.com/excilys/gatling/wiki/Continuous-Integration

Here’s your migrated simulation:

package basic

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
import io.gatling.http.HeaderNames._
import scala.concurrent.duration._

class LoadGen extends Simulation {

val nbUsers = Integer.getInteger(“users”, 1)
val myRamp = java.lang.Long.getLong(“ramp”, 0).toInt
val myIter = Integer.getInteger(“iter”, 1).toInt
val myUrl = System.getProperty(“url”)
val pgReqFeeder = ssv(“pgRequest.ssv”).circular
println(myUrl)
val httpConf = http
.baseURL(myUrl)
.acceptHeader("/")

val scn = scenario(“Enrichment Demo”)
.repeat(myIter) {
feed(pgReqFeeder)
.exec(
http(“Http Post”)
.post("/something.do")
.header(“Content-Type”, “application/json”)
.body(StringBody("${REQUESTBODY}"))
.asJSON
.check(status.is(200)))

}

setUp(scn.inject(rampUsers(nbUsers) over myRamp)).protocols(httpConf)
}

6

How do you get the Start and End time with Gatling?
You are not looking at the beginning and the end of the process, right?

7

@Nicolas: I am using start time and end time at beginning and end of the process. But simulation prints out elapsed time, and reports do show RPS. Start time and end time are not used in any calculations.

@Stéphane: I will try using the latest snapshot today and report results.