Max requests per second far lower than expected

Gatling (Open-Source)
1

Gatling version: 3.13.1 (must be up to date)
Gatling flavor: java kotlin scala javascript typescript
Gatling build tool: maven gradle sbt bundle npm

I read the [guidelines] and [how to ask a question] topics.
I provided a [SSCCE] (or at least, all information to help the community understand my topic)
I copied output I observe, and explain what I think should be.

NOTE: I had to remove the links from above because the system would not let me post this with them included

Hello everyone! I’ve just started exploring Gatling as a way to test a reverse proxy I am building. I have a very simple example simulation which is intended to be a stress test on the system to find the failure point. I am running this using the maven wrapper from my M1 Mackbook Pro.

if I configure Gatling for a slow enough ramp, the simulation is able to run (mostly) smoothly and get up to about 2500 TPS. At this point calls start timing out and it seems like some kind of cascade failure happens as any more users added just also time out. 2,500 tps seems quite low to me as I am able to send 20,000 TPS without issue using both wrk and vegeta.

I have set .shareConnections() on my http protocol which helped dramatically, but I still feel like I must be approaching this the wrong way. My intuition says that I should be having each user send multiple requests rather than 1 request per user to achieve higher request volumes but from the documentation and reading other sources online it seems that 1 user/1 request is the “best practice” for Gatling.

Any input would be greatly appreciated!

ReverseProxySimulation.java

package reverseproxy;

import io.gatling.javaapi.core.*;
import io.gatling.javaapi.http.*;

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;

import java.time.Duration;
import java.util.Iterator;
import java.util.Map;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.Supplier;
import java.util.stream.Stream;

public class ReverseProxySimulation extends Simulation {

    private static final String BASE_URL = "http://192.168.26.20/api";

    private final HttpProtocolBuilder httpProtocol = http.baseUrl(BASE_URL)
            .contentTypeHeader("application/json")
            .shareConnections()
            .asyncNameResolution();

    private static final int INCREMENT = Integer.parseInt(System.getProperty("INCREMENT", "500"));
    private static final int NUM_STEPS = Integer.parseInt(System.getProperty("NUM_STEPS", "4"));
    private static final int STEP_TIME = Integer.parseInt(System.getProperty("STEP_TIME", "60"));
    private static final int STARTING_VOLUME = Integer.parseInt(System.getProperty("STARTING_VOLUME", "2000"));

    private static AtomicInteger txnIdIterator = new AtomicInteger(10000);
    private static Iterator<Map<String, Object>> txnIdFeeder = 
        Stream.generate((Supplier<Map<String, Object>>) () -> {
            return Map.of("txn_id", txnIdIterator.getAndIncrement());
        }).iterator();
    private static FeederBuilder.FileBased<Object> importerUUIDFeeder = jsonFile("data/importer_uuid_feed.json").random();

    private static ChainBuilder sendTxn = feed(txnIdFeeder)
        .feed(importerUUIDFeeder)
        .exec(http("send request")
        .post("/#{importer_uuid}")
        .basicAuth("parse_only_user", "ABCD1234$")
        .body(ElFileBody("bodies/txn_id.json"))
        .requestTimeout(Duration.ofSeconds(1))
        .check(status().is(200))
        .check(jmesPath("\"transaction-results\".\"transaction-id\"").isEL("#{txn_id}")));

    private ScenarioBuilder scn = scenario("reverse proxy test")
            .exec(sendTxn);

    {
        setUp(
            scn.injectOpen(
                // rampUsersPerSec(0)
                // .to(STARTING_VOLUME)
                // .during(Duration.ofMinutes(5)),
                incrementUsersPerSec(INCREMENT)
                .times(NUM_STEPS)
                .eachLevelLasting(STEP_TIME)
                .separatedByRampsLasting(30)
                .startingFrom(STARTING_VOLUME)
            )
        ).protocols(httpProtocol);
    }
}

txn_id.json

{
    "txn_id": #{txn_id}
}

importer_uuid_feeder.json

[
    {
        "importer_uuid": "39e6f3b0-aa69-4b39-9037-0cb08dcf1705"
    },
    {
        "importer_uuid": "7ea50807-0d92-4f1f-8f35-6f04b4b5955b"
    }
]
2

2,500 tps seems quite low to me as I am able to send 20,000 TPS without issue using both wrk and vegeta.
I have set .shareConnections() on my http protocol which helped dramatically

This is certainly what you are actually doing with these other tools. At 20,000 TPS, if you’re not keeping connections alive (and are not playing with dangerous kernel settings), you’re going to run out of ephemeral ports because of TCP TIME_WAIT.

it seems that 1 user/1 request is the “best practice” for Gatling.

It really depends on the real world traffic on your application you’re trying to simulate.

  • are you simulating internet traffic coming from a huge number of individual traffic? => default set up
  • or traffic coming from a limited number of clients keeping connections alive between requests? => shareConnections

At this point calls start timing out and it seems like some kind of cascade failure happens as any more users added just also time out.

Your system under test is likely the one to saturate here, not being able to handle a high connections creation create.

3

Thank you for confirming my expectations :slight_smile:
My api is a b2b platform so I believe that shared connections is the correct configuration for me. Can you suggest any changes I can make to get similar numbers in Gatling that I see in wrk or vegeta? Or potentially point me to some documentation pages that describe how to send multiple requests per virtual user?

4

Or potentially point me to some documentation pages that describe how to send multiple requests per virtual user?

5

Thank you for your help!