Graphite DataWritersStatsEngine start timeout after 5s

Hello,

Since upgrading from Gatling 3.0.5 to 3.5.1, we’re getting the following sporadic error.

12:58:18.594 [ERROR] i.g.c.c.Controller - Actor io.gatling.core.controller.Controller@4d4df077 crashed on message Some(Start(io.gatling.core.scenario.Scenarios@37fff86b)) java.util.concurrent.TimeoutException: Future timed out after [5 seconds] at scala.concurrent.impl.Promise$DefaultPromise.tryAwait0(Promise.scala:212) at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219) at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:104) at scala.concurrent.Await$.$anonfun$ready$1(package.scala:174) at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread$$anon$3.block(ThreadPoolBuilder.scala:174) at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3313) at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread.blockOn(ThreadPoolBuilder.scala:172) at scala.concurrent.Await$.ready(package.scala:124) at io.gatling.core.stats.DataWritersStatsEngine.start(DataWritersStatsEngine.scala:89) at io.gatling.core.controller.Controller$$anonfun$1.applyOrElse(Controller.scala:63) at io.gatling.core.controller.Controller$$anonfun$1.applyOrElse(Controller.scala:54) at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:35) at akka.actor.FSM.processEvent(FSM.scala:804) at akka.actor.FSM.processEvent$(FSM.scala:801) at io.gatling.core.controller.Controller.processEvent(Controller.scala:44) at akka.actor.FSM.akka$actor$FSM$$processMsg(FSM.scala:798) at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:793) at akka.actor.Actor.aroundReceive(Actor.scala:537) at akka.actor.Actor.aroundReceive$(Actor.scala:535) at io.gatling.core.akka.BaseActor.aroundReceive(BaseActor.scala:25) at akka.actor.ActorCell.receiveMessage(ActorCell.scala:577) at akka.actor.ActorCell.invoke(ActorCell.scala:547) at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270) at akka.dispatch.Mailbox.run(Mailbox.scala:231) at akka.dispatch.Mailbox.exec(Mailbox.scala:243) at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289) at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056) at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692) at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

It seems like Graphite initialization times out after 5s from time to time. 3 questions:

1. Does the configuration below look right?

graphite {
#light = false # only send the all* stats
host = “myhost.com” # The host where the Carbon server is located
port = 2004 # The port to which the Carbon server listens to (2003 is default for plaintext, 2004 is default for pickle)
#protocol = “tcp” # The protocol used to send data to Carbon (currently supported : “tcp”, “udp”)
#rootPathPrefix = “gatling” # The common prefix of all metrics sent to Graphite
#bufferSize = 8192 # Internal data buffer size, in bytes
#writePeriod = 1 # Write period, in seconds
}

2. Is there a way to configure this timeout value?

3. Is there any guidance you can provide to get past this issue?

Thank you!

BTW, it’s possible the problem isn’t with Graphite specifically, but rather with all DataWriters (file, console, graphite) init timeout in general. I do see the following line after the exception above:

at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
14:02:23.572 [INFO ] i.g.g.GraphiteDataWriter - Initialized

Bumping