Can I use Gatling to test a custom java action

Gatling (Open-Source)
1

Hi,

I am writing a system which exposes many endpoints which clients access. One use case we have is of an application starting up. This app calls a number of http endpoints sequentially. I want to time how long it takes to complete all of these calls under various loads.

The application uses a java client which is wrapper around the http API, which handles retries in the event receiving a 429 - too many requests.

I have attempted to plug the java client into Gatling. It was a bit awkward and I have some concerns that what I am attempting is not really possible. For example, some of the results are a bit dubious.

My code looks like this:

import static io.gatling.javaapi.core.CoreDsl.atOnce;
import static java.util.Arrays.asList;

import com.company.horizon.clientsdk.Horizon;
import com.company.horizon.clientsdk.HorizonConfig;
import com.company.horizon.clientsdk.criteria.StreamQuery;
import com.company.horizon.clientsdk.subscription.Port;
import com.company.horizon.performance.client.HorizonClientDsl;
import io.gatling.javaapi.core.CoreDsl;
import io.gatling.javaapi.core.ScenarioBuilder;
import io.gatling.javaapi.core.Simulation;

public class ExampleSimulation extends Simulation {

    {
        Horizon horizon = Horizon.create(HorizonConfig.ephemeral(1035), Port.random());
        var productQuery = new StreamQuery("product");
        
        var queryActionBuilder = HorizonClientDsl.horizon(horizon)
            .name("StockPricingAppStartup")
            .queries(asList(productQuery.toSearchQuery()));

        ScenarioBuilder scn = CoreDsl.scenario("StockPricingAppStartupTest")
            .exec(queryActionBuilder);
            
        setUp(
            scn.injectOpen(atOnce(1))
        );
    }

}

import com.company.horizon.clientsdk.Horizon;

public class HorizonClientDsl {
    public static ClientQueryActionBuilder horizon(Horizon horizon) {
        return new ClientQueryActionBuilder(horizon);
    }
}

import com.company.horizon.clientsdk.Horizon;
import com.company.horizon.clientsdk.document.SearchQuery;
import io.gatling.javaapi.core.ActionBuilder;
import io.gatling.javaapi.core.ChainBuilder;
import java.util.List;

public class ClientQueryActionBuilder implements ActionBuilder {

    private ScalaClientQueryActionBuilder scalaClientQueryActionBuilder;

    public ClientQueryActionBuilder(Horizon horizon) {
        this.scalaClientQueryActionBuilder = new ScalaClientQueryActionBuilder();
        scalaClientQueryActionBuilder.setHorizon(horizon);
    }

    @Override
    public io.gatling.core.action.builder.ActionBuilder asScala() {
        return scalaClientQueryActionBuilder;
    }

    @Override
    public ChainBuilder toChainBuilder() {
        return ActionBuilder.super.toChainBuilder();
    }

    public ClientQueryActionBuilder queries(List<SearchQuery> searchQueries) {
        scalaClientQueryActionBuilder.setSearchQueries(searchQueries);
        return this;
    }

    public ClientQueryActionBuilder name(String name) {
        scalaClientQueryActionBuilder.setName(name);
        return this;
    }
}

import com.company.horizon.clientsdk.Horizon;
import com.company.horizon.clientsdk.document.SearchQuery;
import io.gatling.core.action.Action;
import io.gatling.core.action.builder.ActionBuilder;
import io.gatling.core.structure.ScenarioContext;
import java.util.List;

public class ScalaClientQueryActionBuilder implements ActionBuilder {
    
    private Horizon horizon;
    private List<SearchQuery> searchQueries;
    private String name;

    @Override
    public Action build(ScenarioContext ctx, Action next) {
        var statsEngine = ctx.coreComponents().statsEngine();
        return new QueryAction(next, name, horizon, searchQueries, statsEngine);
    }

    public void setHorizon(Horizon horizon) {
        this.horizon = horizon;
    }

    public void setSearchQueries(List<SearchQuery> searchQueries) {
        this.searchQueries = searchQueries;
    }

    public void setName(String name) {
        this.name = name;
    }
}

import com.company.horizon.clientsdk.Horizon;
import com.company.horizon.clientsdk.document.MapItemParser;
import com.company.horizon.clientsdk.document.SearchQuery;
import com.typesafe.scalalogging.Logger;
import io.gatling.commons.stats.KO$;
import io.gatling.commons.stats.OK$;
import io.gatling.core.action.Action;
import io.gatling.core.session.Session;
import io.gatling.core.stats.StatsEngine;
import scala.Option;
import scala.collection.immutable.List;

public class QueryAction implements Action {

    private final Action next;
    private final String name;
    private final Horizon horizon;
    private final java.util.List<SearchQuery> queries;
    private StatsEngine statsEngine;
    private Logger logger;

    public QueryAction(Action next, String name, Horizon horizon, java.util.List<SearchQuery> queries, StatsEngine statsEngine) {
        this.next = next;
        this.name = name;
        this.horizon = horizon;
        this.queries = queries;
        this.statsEngine = statsEngine;
    }

    @Override
    public String name() {
        return name;
    }

    @Override
    public void execute(Session session) {
        List<String> groups = scala.collection.immutable.List.<String>newBuilder().result();
        var parser = new MapItemParser();
        long start = System.currentTimeMillis();
        try {
            for (SearchQuery query : queries) {
                var mapItems = horizon.dao(parser).find(query);
            }
            long end = System.currentTimeMillis();
            statsEngine.logResponse(name, groups, name, start, end, OK$.MODULE$, Option.empty(), Option.empty());
        } catch (Exception e) {
            long end = System.currentTimeMillis();
            statsEngine.logResponse(name, groups, name, start, end, KO$.MODULE$, Option.empty(), Option.apply(e.getMessage()));
        }
        next.execute(session);
    }

    @Override
    public void com$typesafe$scalalogging$StrictLogging$_setter_$logger_$eq(Logger logger) {
        this.logger = logger;
    }

    @Override
    public Logger logger() {
        return logger;
    }
}

This seems to mostly work. However I’ve noticed a few odd things.
a) the minimum time for one of these action is an order magnitude smaller than it should be.
b) the simulation log looks weird. The users start after the requests happen. e.g. this run with two users

RUN com.company.horizon.performance.ExampleSimulation examplesimulation 1707583474903 3.10.3
REQUEST StockPricingAppStartup 1707583475002 1707583481456 OK
REQUEST StockPricingAppStartup 1707583475002 1707583481457 OK
USER StockPricingAppStartupTest START 1707583481483
USER StockPricingAppStartupTest START 1707583481483
USER StockPricingAppStartupTest END 1707583481484
USER StockPricingAppStartupTest END 1707583481484

I’m worried that I am using in a Gatling in a way that is just not right. Or maybe there are a few things I can tweak to make this work.

TL DR; Can I use Gatling to test the performance of custom bits of java code.

If you made to here, thanks for reading :slight_smile:
Tom

2

Hi @Maverick,

Welcome aboard!

First, congratulations! You indeed managed to get so far!

Each time, or the second time? From your implementation, you have only one “horizon” instance shared among users. So, perhaps there is any cache in this client?

The log is an internal that can change any time. (And to be honest, I cannot read it).
But are you sure you understand the keyword? Is “REQUEST” keyword the full request?
I know for sure that Gatling is used to compute time for a lot of different parts (DNS resolution, tcp, then tls negociation, body encoding/decoding). From your extract, it may be about the preflight, the request forgery before actually perform the request (that are traced with “START” and “END”).
Does that make sense? (once again, I cannot read it myself)

As the purpose of gatling is to create performance load testing in a performant way. I doubt that your current “horizon” client manage all this kind of aspects (to simulate different clients per virtual user, but being enough memory optimized to manage a large amount of them).

I hope my interrogation help you find what is wrong (either in your log comprehension or in your implementation). But as you mention, it is a custom client that try to tweak the internal of Gatling. And we (Gatling Corp employees) won’t be able to provide (free) help on that.

Cheers!

3

Hi,

Your QueryAction doesn’t work: it performs blocking operations in the caller thread which is one of the few shared threads the Gatling engine uses to run. When a virtual user is blocking there, it blocks all the other virtual users that share the same thread. If you want to call a blocking API, you must offload to a different thread pool.

4

Thank you so much @sbrevet and @slandelle for your responses.

This endeavour drove me slightly insane but I believe it now works.

@slandelle your solution, seemed to work and the numbers agree with my other tests.

I fixed it with a crude wrapping of a Thread.

import com.company.horizon.clientsdk.Horizon;
import com.company.horizon.clientsdk.document.MapItemParser;
import com.company.horizon.clientsdk.document.SearchQuery;
import com.typesafe.scalalogging.Logger;
import io.gatling.commons.stats.KO$;
import io.gatling.commons.stats.OK$;
import io.gatling.commons.util.Clock;
import io.gatling.core.action.Action;
import io.gatling.core.session.Session;
import io.gatling.core.stats.StatsEngine;
import java.util.List;
import scala.Option;

public class QueryAction implements Action {

    private final Action next;
    private final String name;
    private final Horizon horizon;
    private final List<SearchQuery> queries;
    private final MapItemParser parser;
    private StatsEngine statsEngine;
    private final Clock clock;
    private Logger logger;

    public QueryAction(Action next, String name, Horizon horizon, List<SearchQuery> queries, StatsEngine statsEngine, Clock clock) {
        this.next = next;
        this.name = name;
        this.horizon = horizon;
        this.queries = queries;
        this.statsEngine = statsEngine;
        this.clock = clock;
        parser = new MapItemParser();
    }

    @Override
    public String name() {
        return name;
    }

    @Override
    public void execute(Session session) {
        // Why the new thread?
        // https://community.gatling.io/t/can-i-use-gatling-to-test-a-custom-java-action/8718/3
        Thread thread = new Thread(() -> runQueries(session));
        thread.start();
    }

    private void runQueries(Session session) {
        long start = clock.nowMillis();
        try {
            for (SearchQuery query : queries) {
                horizon.dao(parser).find(query);
            }
            long end = clock.nowMillis();
            statsEngine.logResponse(session.scenario(), session.groups(), name, start, end, OK$.MODULE$, Option.apply("200"), Option.empty());
        } catch (Exception e) {
            long end = clock.nowMillis();
            statsEngine.logResponse(session.scenario(), session.groups(), name, start, end, KO$.MODULE$, Option.empty(), Option.apply(e.getMessage()));
        }
        next.execute(session);
    }

    @Override
    public void com$typesafe$scalalogging$StrictLogging$_setter_$logger_$eq(Logger logger) {
        this.logger = logger;
    }

    @Override
    public Logger logger() {
        return logger;
    }
    
}

I made a change to how groups were being handled (previously I was just passing an empty list),

statsEngine.logResponse(session.scenario(), session.groups(), name, start, end, OK$.MODULE$, Option.apply("200"), Option.empty());

which I thought would make groups work. However when I split the queries into single executables and put in a group e.g. like,

ScenarioBuilder scn = CoreDsl.scenario("StockPricingAppStartupTest")
            .group("AppStartup").on(
                startupStep(pricingModeQuery),
                startupStep(cm2vEnvironmentQuery),
                startupStep(productListingQuery),
                startupStep(exchangeFlagQuery), 
                startupStep(exchangeSettingsQuery),
                startupStep(feedSettingsQuery),
                startupStep(productQuery));
        setUp(
            scn.injectOpen(rampUsers(120).during(Duration.ofSeconds(5)))
        );

The queries execute fine but the groups always report a time of 0. In the simulation.log it looks like this:

REQUEST	AppStartup	productPricingMode	1707772721711	1707772734231	OK	 
REQUEST	AppStartup	product	1707772733746	1707772734232	OK	 
GROUP	AppStartup	1707772721700	1707772734231	0	OK

Whereas the example group test I have e.g. :

        ScenarioBuilder scn = CoreDsl.scenario("StockPricingTest")
            .group("AppStartup").on(
                HttpDsl.http("exampleOne")
                    .get("/example1"),
                HttpDsl.http("exampleTwo")
                    .get("/example2"))
            ;
        setUp(
            scn.injectOpen(rampUsers(4).during(120))
        ).protocols(httpProtocol);

the simulation.log looks like:

REQUEST	AppStartup	exampleOne	1707772262496	1707772264553	OK	 
REQUEST	AppStartup	exampleTwo	1707772264575	1707772269585	OK	 
GROUP	AppStartup	1707772262463	1707772269585	7067	OK

I wondering if it is because my QueryAction executes asynchronously, the group “ends” before any of the results of the recorded, and thus records a time of zero. Hmmm.

If I could mutate the session state to say keep queriesInProgressCount, I could then count down the state when the request ends. Is there way to mutate the session state after my action completes? Then I can block waiting on queriesInProgressCount to reach 0;
Sessions are immutable, and I can’t find a point to way to mutate it in my action.

thanks again,
Tom

5

Instead of spawning a new Thread every time, you should use a ForkJoinPool.

6

I did something very similar, injecting a single threaded executor into the session - I want the queries to be executed sequentially.

        ScenarioBuilder scn = CoreDsl.scenario("StockPricingAppStartupTest")
            .group("AppStartup").on(
                insertExecutor(Executors::newSingleThreadExecutor),
                startupStep(pricingModeQuery),
                startupStep(cm2vEnvironmentQuery),
                startupStep(productListingQuery),
                startupStep(exchangeFlagQuery), 
                startupStep(exchangeSettingsQuery),
                startupStep(feedSettingsQuery),
                startupStep(productQuery),
                exec(session -> {
                    ExecutorService executor = session.get(HorizonClientDsl.EXECUTOR);
                    executor.shutdown();
                    try {
                        executor.awaitTermination(100, TimeUnit.SECONDS);
                    } catch (InterruptedException e) {
                        Thread.currentThread().interrupt();
                    }
                    return session;
                })
                );
        setUp(
            scn.injectOpen(atOnceUsers(1)) // rampUsers(1).during(Duration.ofSeconds(5))
        );
    }

and modified the QueryAction to retrieve the executor to submit onto the executor,

import com.company.horizon.clientsdk.Horizon;
import com.company.horizon.clientsdk.document.MapItemParser;
import com.company.horizon.clientsdk.document.SearchQuery;
import com.typesafe.scalalogging.Logger;
import io.gatling.commons.stats.KO$;
import io.gatling.commons.stats.OK$;
import io.gatling.commons.util.Clock;
import io.gatling.core.action.Action;
import io.gatling.core.session.Session;
import io.gatling.core.stats.StatsEngine;
import java.util.List;
import java.util.concurrent.ExecutorService;
import lombok.extern.slf4j.Slf4j;
import scala.Option;

@Slf4j
public class QueryAction implements Action {

    private final Action next;
    private final String name;
    private final Horizon horizon;
    private final List<SearchQuery> queries;
    private final MapItemParser parser;
    private final StatsEngine statsEngine;
    private final Clock clock;
    private Logger logger;

    public QueryAction(Action next, String name, Horizon horizon, List<SearchQuery> queries, StatsEngine statsEngine, Clock clock) {
        this.next = next;
        this.name = name;
        this.horizon = horizon;
        this.queries = queries;
        this.statsEngine = statsEngine;
        this.clock = clock;
        parser = new MapItemParser();
    }

    @Override
    public String name() {
        return name;
    }

    @Override
    public void execute(Session session) {
        Option<Object> optionalExecutor = session.attributes().get(HorizonClientDsl.EXECUTOR);
        if (!optionalExecutor.isDefined()) {
            statsEngine.logCrash(session.scenario(), session.groups(), name, "Executor not found");
        }
        // Why the new executor?
        // https://community.gatling.io/t/can-i-use-gatling-to-test-a-custom-java-action/8718/3
        ExecutorService executorService = (ExecutorService) optionalExecutor.get();
        executorService.submit(() -> runQueries(session));
        next.execute(session);
    }

    private void runQueries(Session session) {
        long start = clock.nowMillis();
        try {
            for (SearchQuery query : queries) {
                horizon.dao(parser).find(query);
            }
            long end = clock.nowMillis();
            statsEngine.logResponse(session.scenario(), session.groups(), name, start, end, OK$.MODULE$, Option.apply("200"), Option.empty());
        } catch (Exception e) {
            long end = clock.nowMillis();
            statsEngine.logResponse(session.scenario(), session.groups(), name, start, end, KO$.MODULE$, Option.empty(), Option.apply(e.getMessage()));
        }
    }

    @Override
    public void com$typesafe$scalalogging$StrictLogging$_setter_$logger_$eq(Logger logger) {
        this.logger = logger;
    }

    @Override
    public Logger logger() {
        return logger;
    }
    
}

The simulation.log looks like it works. The requests finish, then the group finishes. But still it assigned a time value of zero (I assume that is what the penultimate value is)

RUN	com.company.horizon.performance.StockPricingAppSimulation	stockpricingappsimulation	1707782309810	 	3.10.3
REQUEST	AppStartup	productPricingMode	1707782309936	1707782313161	OK	 
REQUEST	AppStartup	CMv2ProphetEnvironment	1707782313166	1707782313278	OK	 
REQUEST	AppStartup	productListing	1707782313278	1707782313398	OK	 
REQUEST	AppStartup	exchangeFlag	1707782313399	1707782313555	OK	 
REQUEST	AppStartup	exchangeSettings	1707782313555	1707782313595	OK	 
REQUEST	AppStartup	feedSettings	1707782313595	1707782313782	OK	 
REQUEST	AppStartup	product	1707782313782	1707782313879	OK	 
GROUP	AppStartup	1707782309926	1707782313879	0	OK
USER	StockPricingAppStartupTest	END	1707782313882
USER	StockPricingAppStartupTest	START	1707782313883

The group’s start and end time correlate with the start of the first query and the end of the last query, but yet the group’s time is always reported as 0.

So for some reason the requests in the group do not contribute to the group’s time. I’m very stuck. :frowning:

thanks for your help.
Tom

7

I managed to get group data displayed by setting this variable in gatling.conf,

  charting {
    useGroupDurationMetric = true 
   }
8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.