Limiting concurrency

We have a performance issue in our application: Login is very slow and problematic when too many are initiated too fast.

Unfortunately, that makes it hard to get to the rest of the application for performance testing.

What I would like to do is log in one user at a time, as fast as the application allows, and no faster … and no slower.

One way of doing that would be to add some kind of single threaded gating to a segment of my execution chain. A semaphore or something. So, when the Login begins, we “lock” that part of the scenario, so that other users have to wait. When the login process completes, it “unlocks” the block, and the next user to get CPU time can enter the section.

I imagine that there is probably a way in Scala or Java to accomplish that. But I don’t know if that’s really the right way of accomplishing what I want to accomplish. How would you go about accomplishing this kind of thing, if it were you?

If it were me, I would try to fix login :wink:

What you can do is:

  • a shared AtomicBoolean
  • an asLongAs loop that wraps a pause and whose condition is a compareAndSet on the AtomicBoolean
  • an exec(function) after the login phase that reset the AtomicBoolean so another virtual user can exit the asLongAs loop
  • maybe a rendezVous so that all your virtual users can meet after being logged in.
    Get it?

Cheers,

Stéphane

Yes, first choice is to fix the Login. And they are working on that…

But I think I get it. Based on that, I have created this:

`
object Login {
val loginAllowed = new AtomicBoolean( true )

val singleFile =
asLongAs( session => loginAllowed.compareAndSet( true, false ) == false ) {
pause( 1 )
}
.exec( session => session.set( LOGGED_IN, false ) )
.asLongAs( session => session( LOGGED_IN ).as[Boolean] == false ) {
exec( Login.sequence )
}
.exec( session => { loginAllowed.set( true ); session } )

val everyone = singleFile.rendezVous( Config.numUsers )

val sequence = /* … */
}
`

Look about right?

Yes, that’s basically it :slight_smile:

val loginAllowed = new AtomicBoolean( true )

val singleFile =
asLongAs( session => !loginAllowed.compareAndSet( true, false ) ) { // ! instead of == false
pause( 1 )
}
// only allowed user will proceed until here
.exec( Login.sequence )
.exec( session => { loginAllowed.set( true ); session } )

So, something is not right. Maybe you can help me understand what I did wrong. My code compiles, but at startup I get an exception:

Exception in thread "main" java.lang.ExceptionInInitializerError at com.cigna.compass.scenarios.CompassOnlyActions.<init>(CompassOnlyActions.scala:14) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at java.lang.Class.newInstance(Class.java:374) at io.gatling.core.runner.Runner.run(Runner.scala:37) at io.gatling.app.Gatling.start(Gatling.scala:236) at io.gatling.app.Gatling$.fromMap(Gatling.scala:55) at io.gatling.app.Gatling$.runGatling(Gatling.scala:80) at io.gatling.app.Gatling$.runGatling(Gatling.scala:59) at io.gatling.app.Gatling$.main(Gatling.scala:51) at io.gatling.app.Gatling.main(Gatling.scala) Caused by: java.lang.NullPointerException at io.gatling.core.structure.Execs$$anonfun$exec$1.apply(Execs.scala:30) at io.gatling.core.structure.Execs$$anonfun$exec$1.apply(Execs.scala:30) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251) at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105) at io.gatling.core.structure.Execs$class.exec(Execs.scala:30) at io.gatling.core.Predef$.exec(Predef.scala:33) at io.gatling.core.structure.Execs$class.exec(Execs.scala:28) at io.gatling.core.Predef$.exec(Predef.scala:33) at com.cigna.compass.Login$.<init>(Login.scala:19) at com.cigna.compass.Login$.<clinit>(Login.scala) ... 13 more

The scenario itself is pretty simple:

`
class CompassOnlyActions extends StandardSimulation {

run(
scenario(“Load Test - Compass Only”)
.exec( Login.everyone ) // THIS IS LINE 14 - Something wrong with Login.everyone?
.during( Config.testDuration ) {
uniformRandomSwitch(
exec( API.profile ),
exec( API.displaymap ),
exec( API.cards )
)
}
.exec( Logout.sequence )
)

}
`

I shared Login.everyone previously, but here it is again:

`
object Login {

val loginAllowed = new AtomicBoolean( true )

val singleFile =
asLongAs( session => !loginAllowed.compareAndSet( true, false ) ) {
pause( 1 )
}
.exec( session => session.set( LOGGED_IN, false ) )
.asLongAs( session => session( LOGGED_IN ).as[Boolean] == false ) {
exec( Login.sequence )
}
.exec( session => { loginAllowed.set( true ); session } )

val everyone = singleFile.rendezVous( Config.numUsers )

val sequence = …

`

Any suggestions on how to track this down?

You did something that’s very wrong: a forward reference.

You define sequence AFTER calling exec( Login.sequence ) so it’s null at this time.

I got an extra pair of eyes looking at it with me. Why the compiler let this go but the run-time didn’t work is beyond me:

I use Login.sequence in the definition of Login.singleFile, before I have declared Login.sequence. By just moving the definition of Login.sequence up ahead of the definition of Login.singleFile, the error went away.

http://stackoverflow.com/questions/7762838/forward-references-why-does-this-code-compile

IDEs usually warn you about forward references.
I agree, this is something that should be a compiler error, since it usually leads to runtime errors.
It is, fortunately, easy to fix : move the definition, make it a def or make it a lazy val.

The bad news is: It doesn’t work reliably. After between 1 and 5 users do the login, they all get caught in the holding pattern. I’ve got debug statements everywhere, there is not a thread that is stuck in the login process.

I tried tweaking the code slightly, and now it is pretty consistently stuck after the first user. Here’s the modified code:

val loginAllowed = new AtomicBoolean( true ) val singleFile = exec( session => session.set( LOGGED_IN, false ) ) .asLongAs( session => ! session( LOGGED_IN ).as[Boolean] ) { asLongAs( session => ! loginAllowed.compareAndSet( true, false ) ) { pause( 1 ) } .exec( Login.sequence ) // is definitely setting LOGGED_IN to true at the end .exec( session => { loginAllowed.set( true ); session } ) }

I am unlocking the flag correctly, right?

So, this is WEIRD…

I added a println before the unlock. It is not being called.

So I added a println at the end of my Login.sequence. THAT is not being printed. HUH?

`
.exec(
API.displaymap
.silent
.check( status.saveAs( RESPONSE_STATUS ) )
)
.exec( session => {
var newValue = true
if ( session( RESPONSE_STATUS ).as[Int] == 200 ) {
println( "DEBUG: " + session( USER_NAME ).as[String] + “: SUCCESS!” )
} else {
println( "Login failed for user " + session( USER_NAME ).as[String] )
newValue = false
}
session.set( LOGGED_IN, newValue )
})
.exec( session => {
println( “Login Sequence Complete” )
session
})

`

The string “DEBUG: : SUCCESS!” prints but the string “Login Sequence Complete” does not. This is starting to look like a Gatling bug.

Could it be the outer .asLongAs() loop that is keyed on the LOGGED_IN session value is interrupting chain execution?

In the code I provided, there was such a useless double check with this LOGGED_IN flag.
Your problem is that asLongAs default behavior is to exit ASAP, so you’re never setting loginAllowed back to true.

I meant “there wasn’t”

I suspected something like that, so I restructured the code to release the flag before checking if the login was successful. But that didn’t fix it.

So I told my asLongAs loops to not exit ASAP, and now the logins seem to proceed as they should. WHEW.

Mind if I ask what the rationale was for defaulting exitASAP to true?

It was originally asked for the during loop (actually, all loops are implemented on top of asLongAs): loop could actually be during “duration condition + loop content duration” at max.
But then, I wonder why direct asLongAs also default to exitASAP = true. I agree it doesn’t feel natural, as people expect it to behave like a while loop.

I think I’m going to change this before Gatling 2.0 goes final, except if someone raises an argument against it.

No arguments here. I expect loops exit conditions to be evaluated once per iteration, like in other programming languages. :slight_smile:

during will keep on exiting ASAP, there was a strong pressure for this behavior.