Trouble sending results to graphite

Craig_Erdmann · August 19, 2014, 10:51pm

I’m a new gatling user and am attempting to send results to graphite. I’m using Gatling 2.0.0-RC2 running on an ubuntu box (virtual with only Gatling installed).

My Gatling.conf relevant section looks like this:

data {
writers = “console, file, graphite”
reader = file
console {
light = false
}
}
graphite {
host = “x.x.x.x”
port = 2003
light = false
protocol = “tcp”
rootPathPrefix = “gatling”
bucketWidth = 100
bufferSize = 8192
}

On my graphite box, I have:

Carbon.conf

[cache]
LOCAL_DATA_DIR = /opt/graphite/storage/whisper/
USER =
MAX_CACHE_SIZE = inf
MAX_UPDATES_PER_SECOND = 1000
MAX_CREATES_PER_MINUTE = inf
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2003
PICKLE_RECEIVER_INTERFACE = 127.0.0.1
PICKLE_RECEIVER_PORT = 2004
CACHE_QUERY_INTERFACE = 127.0.0.1
CACHE_QUERY_PORT = 7002
LOG_UPDATES = True

storage-schemas.conf

[stats]
pattern = ^stats..*
priority = 100
retentions = 10s:7d,1m:31d,10m:5y

[Gatling stats]
priority = 110
pattern = ^gatling..*
retentions = 1s:6d,10s:60d

[catchall]
pattern = ^.*
priority = 0
retentions = 60s:5y

storage-aggregation.conf

[sum]
pattern = .count$
xFilesFactor = 0
aggregationMethod = sum

[min]
pattern = .min$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = .max$
xFilesFactor = 0.1
aggregationMethod = max

[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average

On the gatling machine, I can issue the command (using the same user under which i run gatling.sh):

echo “gatling.myapp.datapoint2 15 date -d -${value}min +%s” | nc x.x.x.x 2003

This command executes and I can see the results in graphite

However, when i attempt to run gatling.sh and choose the computerdatabase.BasicSimulation simulation, I receive this error:

Simulation computerdatabase.BasicSimulation started…
[ERROR] [08/19/2014 15:31:39.616] [GatlingSystem-akka.actor.default-dispatcher-8] [akka://GatlingSy
stem/user/$y/$a] Connection refused
akka.actor.ActorInitializationException: exception during creation
at akka.actor.ActorInitializationException$.apply(Actor.scala:164)
at akka.actor.ActorCell.create(ActorCell.scala:596)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scal
a:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.(Socket.java:425)
at java.net.Socket.(Socket.java:208)
at io.gatling.metrics.sender.TcpSender.(TcpSender.scala:27)
at io.gatling.metrics.sender.MetricsSender$.newMetricsSender(MetricsSender.scala:23)
at io.gatling.metrics.GraphiteSender.preStart(GraphiteDataWriter.scala:99)
at akka.actor.Actor$class.aroundPreStart(Actor.scala:470)
at io.gatling.core.akka.BaseActor.aroundPreStart(BaseActor.scala:23)
at akka.actor.ActorCell.create(ActorCell.scala:580)
… 9 more

[INFO] [08/19/2014 15:31:39.635] [GatlingSystem-akka.actor.default-dispatcher-2] [akka://GatlingSys
tem/user/$y/$a] Message [io.gatling.metrics.SendMetrics] from Actor[akka://GatlingSystem/user/$y#13
20734734] to Actor[akka://GatlingSystem/user/$y/$a#1979925254] was not delivered. [1] dead letters
encountered. This logging can be turned off or adjusted with configuration settings ‘akka.log-dead-
letters’ and ‘akka.log-dead-letters-during-shutdown’.

The simulation runs fine, except for sending the metrics to Graphite. I’m assuming i have something not configured properly, any ideas?

Thanks,
Craig

Alex_Bagehot · August 20, 2014, 7:04am

Hi Craig,

If there’s no logging to show what ip/port is being refused you can use strace to attach to the process to output what is happening at the system call level ie. What exact connection refused.

Likelihood is your gatling conf not being picked up.
Have you double checked

http://gatling.io/docs/2.0.0-RC2/general/configuration.html?highlight=command#gatling-conf

?

I’m a new gatling user and am attempting to send results to graphite. I’m using Gatling 2.0.0-RC2 running on an ubuntu box (virtual with only Gatling installed).

My Gatling.conf relevant section looks like this:

data {
writers = “console, file, graphite”
reader = file
console {
light = false
}
}
graphite {
host = “x.x.x.x”
port = 2003
light = false
protocol = “tcp”
rootPathPrefix = “gatling”
bucketWidth = 100
bufferSize = 8192
}

On my graphite box, I have:

Carbon.conf

[cache]
LOCAL_DATA_DIR = /opt/graphite/storage/whisper/
USER =
MAX_CACHE_SIZE = inf
MAX_UPDATES_PER_SECOND = 1000
MAX_CREATES_PER_MINUTE = inf
LINE_RECEIVER_INTERFACE = 0.0.0.0
LINE_RECEIVER_PORT = 2003
PICKLE_RECEIVER_INTERFACE = 127.0.0.1
PICKLE_RECEIVER_PORT = 2004
CACHE_QUERY_INTERFACE = 127.0.0.1
CACHE_QUERY_PORT = 7002
LOG_UPDATES = True

storage-schemas.conf

[stats]
pattern = ^stats..*
priority = 100
retentions = 10s:7d,1m:31d,10m:5y

[Gatling stats]
priority = 110
pattern = ^gatling..*
retentions = 1s:6d,10s:60d

[catchall]
pattern = ^.*
priority = 0
retentions = 60s:5y

storage-aggregation.conf

[sum]
pattern = .count$
xFilesFactor = 0
aggregationMethod = sum

[min]
pattern = .min$
xFilesFactor = 0.1
aggregationMethod = min

[max]
pattern = .max$
xFilesFactor = 0.1
aggregationMethod = max

[default_average]
pattern = .*
xFilesFactor = 0.3
aggregationMethod = average

On the gatling machine, I can issue the command (using the same user under which i run gatling.sh):

echo “gatling.myapp.datapoint2 15 date -d -${value}min +%s” | nc x.x.x.x 2003

This command executes and I can see the results in graphite

However, when i attempt to run gatling.sh and choose the computerdatabase.BasicSimulation simulation, I receive this error:

Simulation computerdatabase.BasicSimulation started…
[ERROR] [08/19/2014 15:31:39.616] [GatlingSystem-akka.actor.default-dispatcher-8] [akka://GatlingSy
stem/user/$y/$a] Connection refused
akka.actor.ActorInitializationException: exception during creation
at akka.actor.ActorInitializationException$.apply(Actor.scala:164)
at akka.actor.ActorCell.create(ActorCell.scala:596)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scal
a:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:579)
at java.net.Socket.connect(Socket.java:528)
at java.net.Socket.(Socket.java:425)
at java.net.Socket.(Socket.java:208)
at io.gatling.metrics.sender.TcpSender.(TcpSender.scala:27)
at io.gatling.metrics.sender.MetricsSender$.newMetricsSender(MetricsSender.scala:23)
at io.gatling.metrics.GraphiteSender.preStart(GraphiteDataWriter.scala:99)
at akka.actor.Actor$class.aroundPreStart(Actor.scala:470)
at io.gatling.core.akka.BaseActor.aroundPreStart(BaseActor.scala:23)
at akka.actor.ActorCell.create(ActorCell.scala:580)
… 9 more

[INFO] [08/19/2014 15:31:39.635] [GatlingSystem-akka.actor.default-dispatcher-2] [akka://GatlingSys
tem/user/$y/$a] Message [io.gatling.metrics.SendMetrics] from Actor[akka://GatlingSystem/user/$y#13
20734734] to Actor[akka://GatlingSystem/user/$y/$a#1979925254] was not delivered. [1] dead letters
encountered. This logging can be turned off or adjusted with configuration settings ‘akka.log-dead-
letters’ and ‘akka.log-dead-letters-during-shutdown’.

The simulation runs fine, except for sending the metrics to Graphite. I’m assuming i have something not configured properly, any ideas?

Thanks,
Craig

Pierre_DAL-PRA · August 20, 2014, 11:27am

Hi Craig,

As you have to choose between UDP or TCP with Gatling’s Graphite writer, please check in carbon.conf that you’ve only enabled TCP and not UDP on port 2003.

Cheers,

Pierre

Craig_Erdmann · August 20, 2014, 2:06pm

Thanks for the help. Turns out it was a configuration error. On the documentation page:

http://gatling.io/docs/2.0.0-RC2/graphite/index.html

It shows:

Configuration of Gatling

In $GATLING_HOME/conf/gatling.conf, be sure to :

have the GraphiteDataWriter in the list of writers
have the correct host and port for Graphite

data {
  writers = "console, file, graphite"
  reader = file
}
graphite {
        host = "192.168.56.101"
        port = 2003
        #light = false              # only send the all* stats
        #protocol = "tcp"           # Choose between 'tcp' or 'udp'
        #rootPathPrefix = "gatling"
        #bucketWidth = 100
        #bufferSize = 8192
      }

The graphite node needs to be contained within data, not separate. Therefore, I fixed my config file to look like this, and it now works:

data {
writers = “console, file, graphite”
reader = file
console {
light = false
}

graphite {

host = “x.x.x.x”

port = 2003

light = false

protocol = “tcp”

rootPathPrefix = “gatling”

bucketWidth = 100

bufferSize = 8192

}

}

–Craig

Craig_Erdmann · August 20, 2014, 3:36pm

Created pull request to change documentation:

https://github.com/gatling/gatling/pull/2108

Carlos_Torres1 · August 21, 2014, 8:46pm

Thanks! I had the same problem.

Topic		Replies	Views
Graphite and gatling results discrepancy Gatling (Open-Source)	9	137	September 11, 2014
Graphite output Gatling (Open-Source)	2	364	April 19, 2021
Gatling not sending events on graphite port 2003 Gatling (Open-Source)	1	213	January 9, 2022
Gatling - Influx DB configuration Gatling (Open-Source)	1	314	February 23, 2021
Gatling Influxdb Status Issue Gatling (Open-Source)	0	203	September 16, 2020

Trouble sending results to graphite

Configuration of Gatling

Related topics