Did 2.2.3-SNAPSHOT enable TCP back pressure with AKKA http?

I tried gatling 2.2.3-SNAPSHOT on a test against a super simple akka http app (this app has a throttled actor as backend that simply reply the responded message, this actor is throttled to handle 200 message per second), I setup a gatling test with 3000 users but caped at 300 request / second.
On gatling 2.2.2 I get the following result

Executions Response Time (ms)
Total OK KO % KO Req/s Min 50th pct 75th pct 95th pct 99th pct Max Mean Std Dev

Global Information 35109 28592 6517 19% 193.972 60 5426 8537 35077 35270 60027 9904 12447

This makes sense to me since there are more requests coming in than the backend actor can handle all the messages have to wait in the actor’s throttlers queue.
But when I switch to gatling 2.2.3-SNAPSHOT I got

Executions Response Time (ms)
Total OK KO % KO Req/s Min 50th pct 75th pct 95th pct 99th pct Max Mean Std Dev

Global Information 43774 27336 16438 38% 241.845 0 72 336 1448 2689 3353 302 546

and

All those KOs are

j.n.ConnectException: connection timed out: localhost/127.0.0.1:8081 16338 99.392 %

It seems that Akka http when working with gatling 2.2.3-SNAPSHOT was able to apply TCP back pressure (or something) and automatically avoid flooding the backend actor’s own message queue.

What’s changed since gatling 2.2.2 that might explain this?
Thanks very much!

No, we don’t expect any behavior change in 2.2.3.
Are you sure you don’t see this because you didn’t restart your app after some initial tests so the queue was already super full?
Could you please provide your sample app?

Hi, Stephane,
Thanks for your response. I have tested back and force several times with clean - recompile etc.
I will clean up the code to a minimum repeatable example and send you the github link soon.

Thanks!

Okay I am still extracting the example, but I found another interesting fact, the backpressure like effect only appears when I run the akka http and gatling 2.2.3-snapshot simulation in the same JVM.
If I run gatling 2.2.2, or the AKKA http server in a different jvm, it disappears. Not sure if that would ring a bell for you but I will continue extracting.

Any test involving the load test tool and the system under test on the same host, and all the more in the same JVM, is completely irrelevant!
Loopback behaves completely differently from a real network interface and results you would get with such setup are meaningless.
You REALLY must have them on different hosts.

Stephane,

Thanks for your advice. I understood that running in the same host or even jvm will give me accurate performance results. The “stress” test I setup was more like an end2end test for an backpressure library we are developing. The reason I had it running in the same JVM is to set this end2end test to run on travis. I still think that gatling 2.2.3-snapshot and 2.2.2 behave with such dramatic difference when testing a http service running in the same JVM is puzzling and interesting (because the back-pressure capability it demonstrated is almost perfect, I can’t explain how that happened). But if you disagree and deem that difference as irrelevant, I won’t send you the extracted example then.
Thanks very much for you help! and thanks again for this great Gatling project.

Actually I may have an explanation, and it’s probably involves the akka http setup I had, so nothing much to do with Gatling. I don’t think it’s necessary to bother you with the example any more.
Thanks very much again for your advice and help.

It would be interesting to know your findings.
Then, Gatling didn’t go under dramatic changes between 2.2.2 and 2.2.3. The only thing that could be remotely related is a Netty upgrade (from 4.0.37.Final to 4.0.40.Final) but nothing fancy.
We don’t plan to implement any backpressure in Gatling atm and I’m not sure it would make sense. I feel like one would want the exact opposite: don’t slow down the load injector.