Circular and random feed does not work as expected

Damien_Hardy · December 17, 2012, 5:46pm

Hello,

I try some Gatling simulation against one of my service.
Using a feed TSV file ( photo_queries.tsv ) containing 26 query strings as ${query}
Looks like :
query
/servlet/photo?memberId=00219l2onef1bire&height=35&width=26&ts=1348109861000
/servlet/photo?memberId=0021nwp1f2a9jd4z&height=35&width=26&ts=1339603511000
/servlet/photo?memberId=0021lpsbgdg9nzg&height=60&width=45&ts=1337034533000
[…]

Here is my Simulation scala code :

package photo
import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import com.excilys.ebi.gatling.jdbc.Predef._
import akka.util.duration._
import bootstrap._

class SimulationPhotoLoad extends Simulation {

def apply = {

val photoQueries = tsv(“photo_queries.tsv”).circular

val urlBase = “http://10.40.0.2/”
val httpConf = httpConfig.baseURL(urlBase).hostHeader(“static”)

val scn = scenario(“My scenario”)
.during(1 minutes) {
feed(photoQueries)
.exec(
http(“Photos”)
.get("${query}")
)
.pause(0 milliseconds, 100 milliseconds)
}

List(scn.configure.users(10).protocolConfig(httpConf))
}
}

The report ends with 260 hits (26 * 10) almost reached in the first 10 seconds and waiting for other 110 seconds doing nothing.
change circular by random does nothing for that.

I excepted that queries were hit again and again during the 2 minutes of the tests resulting on much more hits on the plateform.

What is wrong with my code ?

Thank you for your kind answer.

Cheers,

Excilys · December 17, 2012, 6:53pm

Hi,

You can’t pass query parameters in the get method, you have to use queryParam:
https://github.com/excilys/gatling/wiki/HTTP#wiki-query-params

Though, you have to provide parsed data in your feeder.

Could you try it and tell us if it then works as expected?

Stéphane

Damien_Hardy · December 18, 2012, 9:57am

Hi,

Thank you Stéphane.
Things seams better but lots of 400 errors now

10:14:54.614 [WARN ] c.e.e.g.h.a.GatlingAsyncHandlerActor - Request ‘Photos’ failed : Check ‘in’ failed, found 400 but expected Range(200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210)
10:14:55.194 [WARN ] c.e.e.g.h.a.GatlingAsyncHandlerActor - Request ‘Photos’ failed : Check ‘in’ failed, found 400 but expected Range(200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210)
10:14:55.438 [WARN ] c.e.e.g.h.a.GatlingAsyncHandlerActor - Request ‘Photos’ failed : Check ‘in’ failed, found 400 but expected Range(200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210)
10:14:55.937 [WARN ] c.e.e.g.h.a.GatlingAsyncHandlerActor - Request ‘Photos’ failed : Check ‘in’ failed, found 400 but expected Range(200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210)
10:14:57.564 [WARN ] c.e.e.g.h.a.GatlingAsyncHandlerActor - Request ‘Photos’ failed : Check ‘in’ failed, found 400 but expected Range(200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210)

Excilys · December 18, 2012, 10:05am

Hi Damien,

You can lower the log level in logback.xml conf file:
https://github.com/excilys/gatling/wiki/Configuration#wiki-config-files

Cheers,

Stéphane

Damien_Hardy · December 18, 2012, 10:16am

Thank you Stéphane

It seams that parameters are not set most of the time (230 OK queries seams to be constant in my simulation) random policy give the same result:

11:13:20.681 [DEBUG] c.e.e.g.h.a.GatlingAsyncHandlerActor - Request ‘Photos’ failed : Check ‘in’ failed, found 400 but expected Range(200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210)
request was:http://10.40.0.2//servlet/photo GET headers: Host:static
response was:
status=
400 Bad Request
headers=
“Connection=close”; “Content-Encoding=identity”; “Content-Length=971”; “Content-Type=text/html;charset=utf-8”; “Date=Tue, 18 Dec 2012 10:13:20 GMT”; “Server=Apache”; “Vary=Accept-Encoding,User-Agent”

Nicolas_R · December 18, 2012, 10:20am

Would it be possible to see your current simulation script ?

Damien_Hardy · December 18, 2012, 10:22am

Of course

Here it is :

package photo
import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import com.excilys.ebi.gatling.jdbc.Predef._
import akka.util.duration._
import bootstrap._

class SimulationPhotoLoad extends Simulation {

def apply = {

val photoQueries = tsv(“photo_queries.tsv”).random

val urlBase = “http://10.40.0.2/”
val httpConf = httpConfig.baseURL(urlBase).hostHeader(“static”)

val scn = scenario(“My scenario”)
.during(2 minutes) {
feed(photoQueries)
.exec(
http(“Photos”)
.get("${query}")
.queryParam(“memberId”, “${memberId}”)
.queryParam(“height”, “${height}”)
.queryParam(“width”, “${width}”)
.queryParam(“ts”, “${ts}”)
)
.pause(0 milliseconds, 100 milliseconds)
}

List(scn.configure.users(10).protocolConfig(httpConf).ramp(20))
}
}

I transform my feed file as (tsv):
query memberId height width ts
/servlet/photo 0021el2on8f1bire 35 26 1348109861000
/servlet/photo 0021nfp132a9jd4z 35 26 1339603511000
/servlet/photo 0021lpgbzdg9nzg 60 45 1337034533000
/servlet/photo 002okfgh1qp0vs4 40 40 1306580914000

Excilys · December 18, 2012, 11:40am

Would you have, by any chance, set up your text editor so that it
automatically replaces tabs (like those in your tsv file) with spaces?

Damien_Hardy · December 18, 2012, 11:47am

“0x09” is TAB

Damien_Hardy · December 18, 2012, 11:54am

By the way, replacing with coma (",") and changing tsv by csv in scala give the same result.

Excilys · December 18, 2012, 12:21pm

Damn!

I tested it with the latest Gatling 1.4.0-SNAPSHOT (sorry, I’m in a hurry and this version has better logging) and it works perfectly fine.

Do you think you could try it out?
http://repository-gatling.forge.cloudbees.com/snapshot/com/excilys/ebi/gatling/highcharts/gatling-charts-highcharts/1.4.0-SNAPSHOT/

See logback.xml for enabling logs on HTTP requests and responses.

Here’s your migrated simulation:

package photo
import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import com.excilys.ebi.gatling.jdbc.Predef._
import akka.util.duration._
import bootstrap._

class SimulationPhotoLoad extends Simulation {

val photoQueries = tsv(“photo_queries.tsv”).random

val urlBase = “http://10.40.0.2”
val httpConf = httpConfig.baseURL(urlBase)

val scn = scenario(“My scenario”)
.during(2 minutes) {

feed(photoQueries)
.exec(
http(“Photos”)
.get("${query}")
.queryParam(“memberId”, “${memberId}”)
.queryParam(“height”, “${height}”)
.queryParam(“width”, “${width}”)
.queryParam(“ts”, “${ts}”)

)
.pause(0 milliseconds, 100 milliseconds)
}

setUp(scn.users(10).protocolConfig(httpConf).ramp(20))
}

Damien_Hardy · December 18, 2012, 2:40pm

Thanks to Stéphane I clean my feed file from 3 wrong lines that genereta 400 bad requests (don’t know why but this is not subject).

Now I have a feed file of 23 lines and report looks like :

Nicolas_R · December 18, 2012, 3:35pm

Hi,

I just ran a quick test on my machine. I created a web-server that doesn’t do anything (see serve-tornado.py) and used your simulation against it :

serve-tornado.py (461 Bytes)

BasicExampleSimulation.scala (962 Bytes)

photo_queries.tsv (236 Bytes)

Damien_Hardy · December 18, 2012, 4:09pm

Hmm I have the same result than you with the local tornado service.

But if all requests are done in the few 20 seconds what is it waiting for in the rest of the test ? …(#headache)

And why 230 (as 10 users * 23 lines in feed ) ?
You can see the progress of the test in the capture join
and the stats (for 3 minutes of tests):

Requests
Executions
Response Time (ms)
Total OK KO Min Max Mean Std Dev 95th pct 99th pct Req/s
Global Information
230 230 0
160 330 166 23 170 320 1

Photos
230 230 0
160 330 166 23 170 320 1

Nicolas_R · December 18, 2012, 4:32pm

Bizarre.
Have you used the same TSV file in my simulation against the Tornado web server ?

Damien_Hardy · December 18, 2012, 4:56pm

Nope, mine just changing target for the localhost Tornado

Sorry for flooding. I try to give maximum of data to understand.

To complete here is some apachebench :
ab -H “Host: static” -n 500 -c 10 “http://10.40.0.2/servlet/photo?memberId=002qben1yiboxvs&height=40&width=40&ts=1346331406000”
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.40.0.2 (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Finished 500 requests

Server Software: Apache
Server Hostname: 10.40.0.2
Server Port: 80

Document Path: /servlet/photo?memberId=
002qben1yiboxvs&height=40&width=40&ts=1346331406000
Document Length: 1311 bytes

Concurrency Level: 10
Time taken for tests: 16.343 seconds
Complete requests: 500
Failed requests: 0
Write errors: 0
Total transferred: 801500 bytes
HTML transferred: 655500 bytes
Requests per second: 30.59 [#/sec] (mean)
Time per request: 326.858 [ms] (mean)
Time per request: 32.686 [ms] (mean, across all concurrent requests)
Transfer rate: 47.89 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 158 162 63.1 158 1158
Processing: 160 164 39.8 161 791
Waiting: 160 161 1.1 161 164
Total: 317 326 89.8 319 1949

Percentage of the requests served within a certain time (ms)
50% 319
66% 320
75% 321
80% 321
90% 322
95% 322
98% 323
99% 326
100% 1949 (longest request)

So try it with 1 line feed (the same):

query memberId height width ts

/servlet/photo 002qben1yiboxvs 40 40 1346331406000

Nicolas_R · December 18, 2012, 5:00pm

Ok, by using your prod server, I’m able to reproduce
Hope you don’t mind.

I’ll try to have a look asap.

package basic

import com.excilys.ebi.gatling.core.Predef._
import com.excilys.ebi.gatling.http.Predef._
import akka.util.duration._
import bootstrap._

class BasicExampleSimulation extends Simulation {

def apply = {
val urlBase = “http://static9.viadeo-static.com”
val httpConf = httpConfig.baseURL(urlBase)

val scn = scenario(“My scenario”)
.during(2 minutes) {
exec(
http(“Photos”)
.get("/servlet/photo")
.queryParam(“memberId”, “002va4fhz5qsscl”)
.queryParam(“height”, “40”)
.queryParam(“width”, “40”)
.queryParam(“ts”, “1293559910000”)
)
}
List(scn.configure.users(2).protocolConfig(httpConf).ramp(20))
}
}

Nicolas_R · December 18, 2012, 9:15pm

Found it !

The responses from your server have these headers :

“Accept-Ranges=bytes”;

“Age=0”;

"Cache-Control=public, max-age=2592000";

“Connection=Keep-Alive”;

“Content-Length=1137”;

“Content-Type=image/jpeg”;

“Date=Tue, 18 Dec 2012 21:09:18 GMT”;

“Expires=Thu, 17 Jan 2013 21:09:18 GMT”;

“Last-Modified=Tue, 28 Dec 2010 18:11:50 GMT”;

“Server=Apache”

By default, Gatling take these directives into account. So, in your case, you should define the httpConfig like that :

val httpConf = httpConfig

.baseURL(baseURL)

.disableCaching

Excilys · December 19, 2012, 8:30am

@Nico Nice job! I think someone also ran into the same puzzle some time
ago, so we should write something down on this topic in the FAQ.
@Damien As Nicolas stated, you can disable this feature, but IMHO, if you
want to simulate real users behavior, you'd better keep the default set up
and consider the behavior is the expected one.

Cheers,

Stéphane

Damien_Hardy · December 19, 2012, 10:26am

Great ! Thank you every one.
With cache disabled it work as I was excepting it (like jmeter does)
Now I understand my mistake.

Thank you.