Gatling JDBC third party plugin crashing because of: Failed to start the native thread for java.lang.Thread

Gatling (Open-Source)
1

I have created a simulation for testing jdbc load on database to test Postgres cluster.

The test starts crashing because of problems with creating threads:

[110.738s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.738s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21677"

I am running the test on a virtual machine with plenty resources (8 CPU cores, 32GB RAM), Red Hat 9 and OpendJDK 17.0.12.

The test crashes when the process is using around 21k of threads and 1GB RAM.

The system can use up to 450k threads and limit for process is over 150k of threads.

Any suggestion is welcomed.

Simulation

The simulation is pretty simple:

setUp(
        PgbenchSimulation.SCN.injectOpen(
            nothingFor(ofSeconds(5)),
            rampUsersPerSec(0).to(100).during(ofSeconds(20)),
            constantUsersPerSec(100).during(ofSeconds(10)),
            rampUsersPerSec(100).to(250).during(ofSeconds(10)),
            constantUsersPerSec(250).during(ofSeconds(10)),
            rampUsersPerSec(250).to(500).during(ofSeconds(20)),
            constantUsersPerSec(500).during(ofSeconds(10)),
            rampUsersPerSec(500).to(1000).during(ofSeconds(30)),
            constantUsersPerSec(1000).during(ofSeconds(10)),
            rampUsersPerSec(1000).to(1500).during(ofSeconds(30))
        )
    ).protocols(dataBase);
2

Dependencies (pom.xml):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.broadcom.wt</groupId>
    <artifactId>postgres-ha</artifactId>
    <version>1.0-SNAPSHOT</version>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>io.gatling.highcharts</groupId>
            <artifactId>gatling-charts-highcharts</artifactId>
            <version>3.12.0</version>
        </dependency>
        <dependency>
            <groupId>com.typesafe</groupId>
            <artifactId>config</artifactId>
            <version>1.4.3</version>
        </dependency>
        <dependency>
            <groupId>org.galaxio</groupId>
            <artifactId>gatling-jdbc-plugin_2.13</artifactId>
            <version>0.12.0</version>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>42.7.4</version>
        </dependency>
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
            <version>3.15.0</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.34</version>
            <scope>provided</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>io.gatling</groupId>
                <artifactId>gatling-maven-plugin</artifactId>
                <version>4.9.6</version>
<!--                https://devqa.io/gatling-maven-performance-test-framework/-->
                <configuration>
                    <simulationClass>com.broadcom.wt.pgbench.PgbenchSimulation</simulationClass>
                    <jvmArgs>
                        <jvmArg>-Denv=${env}</jvmArg>
                        <jvmArg>-Dusers=${users}</jvmArg>
                        <jvmArg>-Drampup=${rampup}</jvmArg>
                        <jvmArg>-Dduration=${duration}</jvmArg>
                        <jvmArg>-Dthroughput=${throughput}</jvmArg>
                    </jvmArgs>
                    <propagateSystemProperties>true</propagateSystemProperties>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
3

Log

...

================================================================================
2024-10-07 15:26:38 GMT                                     100s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=12269  KO=0     )
> pgbenchTransaction                                       (OK=12269  KO=0     )

---- Pgbench scenario ----------------------------------------------------------
[##########--------------                                                  ] 13%
          waiting: 60632  / active: 15849  / done: 12269
================================================================================


================================================================================
2024-10-07 15:26:43 GMT                                     105s elapsed
---- Requests ------------------------------------------------------------------
> Global                                                   (OK=13070  KO=0     )
> pgbenchTransaction                                       (OK=13070  KO=0     )

---- Pgbench scenario ----------------------------------------------------------
[##########----------------                                                ] 14%
          waiting: 56675  / active: 19005  / done: 13070
================================================================================

[110.725s][warning][os,thread] Attempt to protect stack guard pages failed (0x00007fe9c5ab0000-0x00007fe9c5ab4000).
#
# A fatal error has been detected by the Java Runtime Environment:
# Native memory allocation (mprotect) failed to protect 16384 bytes. Error detail: memory to guard stack pages
# An error report file with more information is saved as:
# /tmp/postgres-ha/hs_err_pid276528.log
[110.726s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.726s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21667"
[110.728s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.728s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21668"
[110.729s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.729s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21669"
15:26:47.440 [gatling-1-3] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
15:26:47.441 [gatling-1-4] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.730s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
15:26:47.442 [gatling-1-5] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.731s][warning][os,thread] Failed to start the native thread for java.lang.Thread "Logging-Cleaner"
[110.731s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.731s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21670"
[110.731s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
15:26:47.443 [gatling-1-6] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.732s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21671"
[110.732s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
15:26:47.443 [gatling-1-7] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.732s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21672"
15:26:47.444 [gatling-1-8] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
4

Log (finish)

[110.733s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.733s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21673"
15:26:47.445 [gatling-1-9] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.734s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.734s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21674"
15:26:47.446 [gatling-1-10] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.735s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.735s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21675"
15:26:47.447 [gatling-1-11] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
[110.737s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.737s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21676"
15:26:47.449 [gatling-1-12] ERROR o.g.g.jdbc.actions.DBRawQueryAction - Gatling crashed: j.l.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
        at java.base/java.lang.Thread.start0(Native Method)
        at java.base/java.lang.Thread.start(Thread.java:809)
        at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:945)
        at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
        at scala.concurrent.impl.ExecutionContextImpl.execute(ExecutionContextImpl.scala:21)
        at scala.concurrent.impl.Promise$Transformation.submitWithValue(Promise.scala:429)
        at scala.concurrent.impl.Promise$DefaultPromise.submitWithValue(Promise.scala:338)
        at scala.concurrent.impl.Promise$DefaultPromise.dispatchOrAddCallbacks(Promise.scala:312)
        at scala.concurrent.impl.Promise$DefaultPromise.map(Promise.scala:182)
        at scala.concurrent.Future$.apply(Future.scala:687)
        at org.galaxio.gatling.jdbc.db.JDBCClient.connectionResource(JDBCClient.scala:46)
        at org.galaxio.gatling.jdbc.db.JDBCClient.statementResource(JDBCClient.scala:64)
        at org.galaxio.gatling.jdbc.db.JDBCClient.executeRaw(JDBCClient.scala:105)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$4(DBRawQueryAction.scala:26)
        at scala.runtime.java8.JFunction1$mcVJ$sp.apply(JFunction1$mcVJ$sp.scala:18)
        at io.gatling.commons.validation.Success.map(Validation.scala:41)
        at io.gatling.commons.validation.Validation.map$mcJ$sp(Validation.scala:28)
        at io.gatling.commons.validation.Validation.map$mcJ$sp$(Validation.scala:28)
        at io.gatling.commons.validation.Success.map$mcJ$sp(Validation.scala:40)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$3(DBRawQueryAction.scala:21)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$2(DBRawQueryAction.scala:20)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$anonfun$execute$1(DBRawQueryAction.scala:19)
        at io.gatling.commons.validation.Success.flatMap(Validation.scala:42)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.execute(DBRawQueryAction.scala:18)
        at io.gatling.core.action.Action.$bang(Action.scala:42)
        at io.gatling.core.action.Action.$bang$(Action.scala:39)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.io$gatling$core$action$ChainableAction$$super$$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.action.ChainableAction.$bang(Action.scala:75)
        at io.gatling.core.action.ChainableAction.$bang$(Action.scala:73)
        at org.galaxio.gatling.jdbc.actions.DBRawQueryAction.$bang(DBRawQueryAction.scala:12)
        at io.gatling.core.controller.inject.Workload.startUser(Workload.scala:53)
        at io.gatling.core.controller.inject.Workload.$anonfun$injectUser$2(Workload.scala:66)
        at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
        at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:156)
        at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:405)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:840)
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007ff074b99000, 65536, 1) failed; error='Not enough space' (errno=12)
[thread 276543 also had an error]
[110.738s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.738s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21677"
[110.739s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.739s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21678"
[110.740s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.740s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21679"
[110.741s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.741s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21680"
[110.742s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.742s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21681"
[110.743s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.743s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21682"
[110.761s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.761s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21683"
[thread 278742 also had an error]
[110.762s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.763s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21685"
[thread 298325 also had an error]
[110.764s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.764s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21687"
[thread 282110 also had an error]
[110.765s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.765s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21689"
[thread 282091 also had an error]
[110.766s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.767s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21691"
[thread 282113 also had an error]
[110.768s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.768s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21693"
[thread 282132 also had an error]
[110.769s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.770s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21695"
[thread 282095 also had an error]
[110.770s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.771s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21697"
[thread 297825 also had an error]
[110.772s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.772s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21699"
[thread 282121 also had an error]
[110.773s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.773s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21701"
[thread 276705 also had an error]
[110.774s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.775s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21703"
[thread 282004 also had an error]
[110.775s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.776s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21705"
[thread 282131 also had an error]
[110.776s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.777s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21707"
[thread 278753 also had an error]
[110.777s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.778s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21709"
[thread 297201 also had an error]
[110.779s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.779s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21711"
[thread 297893 also had an error]
[110.779s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.780s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21713"
[thread 298252 also had an error]
[110.781s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.781s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21715"
[thread 298323 also had an error]
[110.782s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.782s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21717"
[thread 282128 also had an error]
[110.783s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.783s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21719"
[thread 298182 also had an error][110.784s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.

[110.784s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21720"
[thread 278740 also had an error]
[110.785s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.785s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21723"
[thread 282137 also had an error]
[110.786s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.787s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21725"
[thread 298326 also had an error]
[110.787s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
[110.788s][warning][os,thread] Failed to start the native thread for java.lang.Thread "pool-3-thread-21727"
[thread 278689 also had an error]
#
# If you would like to submit a bug report, please visit:
#   https://access.redhat.com/support/cases/
#
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:54 min
[INFO] Finished at: 2024-10-07T15:26:50Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal io.gatling:gatling-maven-plugin:4.9.6:test (default-cli) on project postgres-ha: Gatling failed.: ForkException -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
5

Hello,

Sorry, but this forum is only for official Gatling components.
The org.galaxio:gatling-jdbc-plugin plugin is a third party plugin developed here: GitHub - galax-io/gatling-jdbc-plugin: Simple gatling plugin for JDBC

You might want to directly reach out to its author there.

The test crashes when the process is using around 21k of threads

I guess this plugin spawns 1 native thread per concurrent JDBC operations, which would mean that you’re trying to generate too much load and your database can’t keep up and concurrent operations pile up in memory.
But that’s for the plugin author to confirm.

Regards

6

I have created a GitHub issue for the plugin: Gatling crashing because of: Failed to start the native thread for java.lang.Thread · Issue #6 · galax-io/gatling-jdbc-plugin · GitHub

7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.