Active users grow with my custom protocol implementation and constantUsersPerSec


I implemented an Apache Kafka 0.8 protocol support using Gatling 2.1.3:

However, the graph in an HTML report shows that active users grow even when I use constantUsersPerSec as the attached image file
though the number of threads in a java process doesn’t grow so much during a stress test.

The simulation file I used is as follows:

And the code to send requests to Apache Kafka 0.8 is as follows:

Am I doing something wrong? If so, how to fix bugs in my code?

Thanks in advance.


Your issue is that you’re blocking on the send Futures:

As a results, you have a thread starvation, and users pile up in the starting queue without getting a chance to run.

As KafkaProducer.send returns regular Futures and not CompletableFutures so you can react on completion instead of blocking, you should use instead the version that takes Callback:

Get it?


The number of active users still grows with a callback, but the growth rate becomes smaller:

2015年1月29日木曜日 9時24分13秒 UTC+9 Stéphane Landelle:

Are you sure creating and closing a producer on each event is the proper pattern? Is this how Kafka is supposed to be used? Shouldn’t such component be reused and shared?

Thank you for your advice.
The issue in my implementation is solved.

2015年1月30日金曜日 15時39分35秒 UTC+9 Stéphane Landelle: