Gatling performance test causing tons of sockets in CLOSE_WAIT

Hi folks,

I was helping a colleague with a Gatling script to do some pre-production testing - basically sending some JSON over a REST API as part of a server to server communication

Initially we were puzzled that Gatling reported a few long-runners not shown in the server’s access.log - re-running the same tests with JMeter showed that the long-runners are caused by the time of connection to the server (this is not directly visible in Gatling AFAIK).

So we had a look at the numbers of connections being used on the load injector box and came across many connections in CLOSE_WAIT (might explain the behaviour above)

  • Each & every request seems to leave one socket in CLOSE_WAIT (~5.000 after a short test run)

  • The script starts a user & session and only executes a single request so we create a lot of session and virtual users (~5.000)

  • We did compare the sockets being used with “Apache AB” and that test run leaves no connections behind so the problem stems from the Gatling script
    We set “keepAlive=false” in gatling.conf which IMHO should do the trick but it had not the desired effect

  • we left a lot fewer connection behinds but still growing (probably 80% less but I have no hard number yet)

  • we still don’t see the behaviour of “Apache AB” where 20 workers used a maximum of 21 sockets

  • the test run suddenly had a lot more long-runners

Having said that I’m a bit puzzled - what I’m doing wrong in terms go HTTP connections and sockets?

Thanks in advance,

Siegfried Goeschl

PS: My colleague is using Windows (he was given little choice) and Gatling 2.2.3 - not perfect :frowning:

I’m sorry, but I’m afraid we can’t do anything without a way to reproduce your issue (with up-to-date Gatling version).

Hi Stéphane,

understood :slight_smile: - my colleague moved to 2.3.0 already to no avail. I’m teaming up with him tomorrow to provide more details to reproduce the problem

Thanks in advance,

Siegfried Goeschl

Good luck :slight_smile:

Also note that antivirus on Windows are rarely load testing friendly…

The follow up - sorry for the dealy but I was sick in the mean time :-()

IP Address

Please note that this is more or less an academic question for my problem

  • The test setup for a virtual user only issuing a single request probably does not reflect reality (but I need to double-check)
  • I’m still puzzled with the current implementation of “keep-alive” but I also need to double-check hitting the test server
    But I still want to provide a complete answer for my colleague :slight_smile:

Sorry, can’t reproduce.

Hi Stephane,

that is somehow surprising since this would be the expected behaviour - having said that it depends on the operating system, gatling.conf and kernel settings.

In my case running on Mac OS X 10.13.1 with the official Gatling download

gatling-charts-highcharts-bundle-2.3.0> sw_vers

ProductName: Mac OS X

ProductVersion: 10.13.1

BuildVersion: 17B48

gatling-charts-highcharts-bundle-2.3.0> java -version

java version “1.8.0_121”

Java™ SE Runtime Environment (build 1.8.0_121-b13)

Java HotSpot™ 64-Bit Server VM (build 25.121-b13, mixed mode)

gatling-charts-highcharts-bundle-2.3.0> ./bin/gatling.sh --simulation computerdatabase.advanced.AdvancedSimulationStep05

gatling-charts-highcharts-bundle-2.3.0> netstat -a -n | grep 35.158.229.206

tcp4 0 0 192.168.0.10.60067 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60069 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60070 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60068 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60071 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60072 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60074 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60075 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60076 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60073 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60077 35.158.229.206.80 TIME_WAIT

tcp4 0 0 192.168.0.10.60078 35.158.229.206.80 TIME_WAIT

MacBook-Pro-de-slandelle:bin slandelle$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.11.6
BuildVersion: 15G17023

MacBook-Pro-de-slandelle:bin slandelle$ java -version
java version “1.8.0_152”
Java™ SE Runtime Environment (build 1.8.0_152-b16)
Java HotSpot™ 64-Bit Server VM (build 25.152-b16, mixed mode)

MacBook-Pro-de-slandelle:bin slandelle$ ./gatling.sh --simulation computerdatabase.advanced.AdvancedSimulationStep05

MacBook-Pro-de-slandelle:bin slandelle$ netstat -a -n | grep 35.158.229.206
MacBook-Pro-de-slandelle:bin slandelle$

¯_(ツ)_/¯

Hi Stephane,

two question

  • is your test server mapped to the same IP address as shown below?
  • did you check the connections immediately

nslookup computer-database.gatling.io

Server: 195.34.133.21
Address: 195.34.133.21#53

Thanks in advance,

Siegfried Goeschl

* is your test server mapped to the same IP address as shown below?

MacBook-Pro-de-slandelle:async-http-client-project slandelle$ nslookup
computer-database.gatling.io
Server: 8.8.4.4
Address: 8.8.4.4#53

Non-authoritative answer:
Name: computer-database.gatling.io
Address: *35.158.229.206*

* did you check the connections immediately

Yes, of course