Number of active users constantly increse but no queues build up

We are using a network of brokers as described in (http://activemq.apache.org/networks-of-brokers.html).

In our test we have setup three AMQ configurations:
lazy val jmsConfa = jms
.connectionFactoryName(“ConnectionFactory”)
.url(amqServera)
.credentials(amqUsr, amqPwd)
.disableAnonymousConnect
.contextFactory(classOf[ActiveMQInitialContextFactory].getName)
.listenerCount(1)
.receiveTimeout(30000)
.usePersistentDeliveryMode
.messageMatcher(IdentificationMatcher)
lazy val jmsConfb = jms
.connectionFactoryName(“ConnectionFactory”)
.url(amqServerb)
.credentials(amqUsr, amqPwd)
.disableAnonymousConnect
.contextFactory(classOf[ActiveMQInitialContextFactory].getName)
.listenerCount(1)
.receiveTimeout(30000)
.usePersistentDeliveryMode
.messageMatcher(IdentificationMatcher)
lazy val jmsConfc = jms
.connectionFactoryName(“ConnectionFactory”)
.url(amqServerc)
.credentials(amqUsr, amqPwd)
.disableAnonymousConnect
.contextFactory(classOf[ActiveMQInitialContextFactory].getName)
.listenerCount(1)
.receiveTimeout(30000)
.usePersistentDeliveryMode
.messageMatcher(IdentificationMatcher)

They are identical, the only thing that is different is the url wich connects to different AMQ brokers in the same network.

Our listeners use the following message matcher.
object IdentificationMatcher extends JmsMessageMatcher {
override def prepareRequest(msg: Message): Unit = {}

override def requestMatchId(msg: Message): String = {
return msg.getStringProperty(“GIMTrackingId”)
}

override def responseMatchId(msg: Message): String = {
return msg.getStringProperty(“ID”)
}
}

In our Simulation we use:

val jmsLoad = scenario(“JMS Load AMQA”).
repeat(1) {
JmsScenarios.createAndPostJmsMessage
}
val jmsLoadb = scenario(“JMS Load AMQB”).
repeat(1) {
JmsScenarios.createAndPostJmsMessage
}
val jmsLoadc = scenario(“JMS Load AMQC”).
repeat(1) {
JmsScenarios.createAndPostJmsMessage
}

And then finally inject like so:
setUp(
jmsLoad.inject(
constantUsersPerSec(jmsMessagesPerSecond).during(loadTime)).
protocols(JSMConfig.jmsConfa),
jmsLoadb.inject(
constantUsersPerSec(jmsMessagesPerSecond).during(loadTime)).
protocols(JSMConfig.jmsConfb),
jmsLoadc.inject(
constantUsersPerSec(jmsMessagesPerSecond).during(loadTime)).
protocols(JSMConfig.jmsConfc)
).
maxDuration(loadTime + 10).
assertions(global.successfulRequests.percent.greaterThan(99),
global.responseTime.max.lessThan(60000))

However this is not working as expected, this is the problems we are having:

  1. During the test number of active users are increasing BUT there are no queues building up, so the only explanation we can find is that the topic reply gets lost so that gatling never gets them.

  2. When the test ends and there are still active users they are not reported as errors by gatling, we would like to make sure that when a test ends there are no active users left.

  3. Is there a way to use gatling as only a load generator, that is have gatling NOT care about replys on the topic, this would help since now the number of active user increse untill the load test crashes with out of memory when we reach around 10000active users.

This is how we post messages:

val postJmsMessage = exec(
  jms("Incoming")
    .reqreply
    .queue(JSMConfig.inQueueName)
    .replyDestination(topic(JSMConfig.replyToTopic)).selector("TYPE='withSuccess'")
    .textMessage("${xmlMessage}")
    .property(Header.MSG_VERSION.jms(), "2.0")
    .property(Header.MSG_TYPE.jms(), "REPORT")
    .property(Header.TRACKING_ID.jms(), "${trackid}"))

My theory is as follows. When gatling sends message to AMQ-A then AMQ-A distribute the message to AMQ-B, now our component consumes the message from AMQ-B and sends the reply to topic on AMQ-B, this results that gatling will never get a reply on the message sent on AMQ-A since this message will be consumed by the gatling JMS listener on AMQ-B wich will not find a matching tracking id.

What we would need to solve this is a common message matcher, that is when ANY of the gatling AMQ scenarios pick a message on the topic it try to match that message will all messages sent by any of the AMQ schenarios. How does this work ATM in gatling? What does gatling do if tracking id does not match? Does it just ignore the message?

Gating doesn’t support cross virtual users transactions atm.

Is it possible then to have gatling not build up the number of active users since this causes gatling to crash when we do long endurance tests. What we would like then is to use gatling as a load generator and then use grafana to monitor our component.
For example can i trick gatling to always match the response message?