Hello,
We are doing a load test on the distributed system using Gatling.
We send a request to Rest endpoint, which sends a bunch of messages to Kafka Topic for the consumers to process it.
One approach we are doing is to send 1000 request to the endpoint and want to check the Kafka Consumer lag using custom code, once all the messages are processed i.e lag is 0 will end the simulation.
how do I implement this in Gatling?
My First cut is
// First Scenario to post to Http Endpoint
I’m using Gatling 2.3.0 along with Spring Boot.
Did you get this working? I am doing something very similar (gatling v2.3) and I found this example helpful:
https://github.com/softmentor/gatling-examples/blob/master/gatling-custom-protocol-demo/src/test/scala/custom/protocol/test/PerfCustomProtocolSimulation.scala
My test is able to measure the throughput I computed with my version of a kafka lag monitor to the console. I don’t write it to the actual gatling report. I don’t think that the DataWriter.dispatch from the example works with 2.3. Not sure the that sort of thing is even supported in free Gatling.
-CH
There’s a community (eg unsupported) plugin for Kafka: https://gatling.io/docs/current/extensions/
No idea what it’s worth or it’s actually maintained.
We’re considering having one for FrontLine.
Thanks for the link. The plugin serves a different use case than the one I’m monitoring kafka for, but could come in handy for other scenarios.
-CH
I am trying to do something similar, could you find a solution for this ?
Ankita,
We use the KafkaConsumer to check lag on all the partitions every 5s. When lag drops to zero, we report the throughput via console logs. I do not post this result to the Gatling report.
Hope this helps,
-CH