How to cache data across virtual users

[jarrowwx]

Our restful service has the concept of a client access token, and a user access token. If a virtual user wants to log in as a particular client, there is a process for getting that token. Once I have it, I would like to cache it and just re-use it for any subsequent requests to “log in” as that client, because that is what our service expects.

In theory, all I need to do is store a Map where the key is the unique elements (client ID or username), and the value is the token. If the map key does not exist, do the normal token getting process, then store the token in the map.

To implement the theory requires some scala knowledge that is a bit above beginner-level. In other words, a little over my level of experience.

I will need to create a tokenCache variable to store the map.
When I need to test for the existence of a key, first I will need to lock the whole map.
Once I determine that a key does NOT exist, (while the map is locked) I create that key, then lock that key, then unlock the map, get the token, store it as the key/value, then unlock that key.
Or, if I determine that the key does exist, I unlock the map, then lock the key (otherwise, I may create massive bottleneck for other users if I just happen to try to lock the key while the token fetching process is in progress), and once the lock is obtained, return the token

Does my logic look right to you?

And which scala constructs would you recommend for something like this? Some good things to google for would be greatly appreciated, since I’m sure that a simple sample class might be asking too much. But if you happen to know of some sample code that implements this kind of functionality, I’d love the pointer to it.

Thanks in advance!

[Excilys]

Sorry, I’d like to get it right. You want several virtual users to be able to log in as the same client, right? Why would you want to do that? Does a client represent a kind of company account that multiple company employees are able to use?

[jarrowwx]

You understand it correctly.

A client token represents, for example, a batch process running on some server farm somewhere. It is SUPPOSED to have all of its separate processes use the same token. I emphasize SUPPOSED because that has not always been the case. But I’d like to have my code behave the way it is supposed to. Because if I don’t, then at some time in the future the call to get a new token will invalidate the old token, rendering other running virtual users inoperable.

To make my life easier, it makes sense to just cache the token. But if there is a better way, I’m open to that, too.

[Excilys]

If possible, the best solution would be to create all your tokens upfront so they’re already there when you start.

If not, writing a cache is something hard enough so you don’t want to write one yourself.
Have you tried Guava?

[jarrowwx]

Part of the scenario, the load characteristics, is token generation. As new users come online, the first thing they do is generate a token. The token is good for a day, so they will do it once per 24 hour period. If they come back in an hour, they will skip token generation. I have a pool of 10k users to pull from, obviously I can’t generate the tokens in advance.

I have not looked at Guava. I didn’t know where to begin my search, that’s why I reached out.

[jarrowwx]

Any thoughts why I shouldn’t just use ConcurrentHashMap, as mentioned here: https://twitter.github.io/scala_school/concurrency.html

[Excilys]

Oh yes I do! You can write your own cache on top of ConcurrentHashMap, of course, but you have to know what your doing.

Using putIfAbsent(key, value) is not straightforward, because you don’t want to compute the value in case the key is already there: that’s exactly the reason why you want a cache in the first place.

If you wrap your ConcurrentHashMap into a Scala Map, you’ll be tempted to use getOrElseUpdate but… it’s currently NOT THREADSAFE: https://issues.scala-lang.org/browse/SI-7943. Fix is planned for Scala 2.11.5 which should be the landing version for Gatling 2.1.

[jarrowwx]

Thanks for that warning.

It occurs to me that I have an inherent problem: I need the result of a Gatling Chain to be put into the map. That chain is going to take anywhere from 1 to 4 requests to complete, and on average takes around 2 seconds to complete.

During that time period, I need to block any other requests to get a token for the same credentials. But I need to allow any requests for any other credentials.

But, if two threads arrive at exactly the same time, both looking for a token “xyz” and see that it is not there, both will try to obtain a token. I need to prevent that.

An appropriate cache for my use case might look like this:

public methods

`

• createLocked( key ) // returns true if key does not exist (and locks it), false otherwise
• put( key, value ) // verifies lock, stores a value AND removes the lock
• get( key ) // Blocks until key exists and is not locked, then returns the value
  `

usage

.doIf( session => tokenCache.createLocked( key ) ) { exec( getToken...saveAs( TOKEN ) ) .exec( session => { tokenCache.put( key, session( TOKEN ).as[String] ); session }) } .exec( session => session.set( TOKEN, tokenCache.get( key ) )

Because of this usage pattern, I have my doubts that an out-of-the-box solution will work for me. But I’d love to be wrong…

So if I had to implement it myself:

Internally, I will need to store two hash maps. One for the tokens, one for the key-level locks. Whenever interacting with the cache, synchronize on the tokenMap object.

In pseudo-code:

`
createLocked ( key ) {
result = false
synchronize on tokenMap {
if ( ! tokenMap.has( key ) )
tokenMap.add( key, “” )
lockMap.add( key, Thread.ID )
result = true
}
return result
}

put ( key, value ) {
synchronize on tokenMap {
if ( ! tokenMap.has( key ) ||
! lockMap.has( key ) ||
lockMap.get( key ) != Thread.ID )
throw( “You have to createLocked(key) before you can put a value into it!” )
tokenMap.update( key, value )
lockMap.remove( key )
}
}

get ( key ) {
successful = false
value = uninitialized
do {
synchronize on tokenMap {
if ( tokenMap.has( key ) && ! lockMap.has( key ) ) {
value = tokenMap.get( key )
successful = true
}
}
if ( ! successful ) then sleep 100ms
} while ( ! successful )
return value
}

`

As of this point, the API is missing a way to update a value, but I’m okay with that.

If you have any comments, I’d love to hear them.

[Excilys]

Sorry, I hadn’t realized you wanted to use Gatling to fetch the tokens.

Then, I think you can solve your problem with 2 ConcurrentHashMaps: one for tokens, and one for fetching permission:

• fetchingPermission.putIfAbsent(tokenKey, true):
• if null: user is the first one => fetch token, then put in tokens map
• if not null: token could be either already fetched, or in the process => loop until tokens.get returns a non null token (1 sec pauses or something)

[Sajjad_Lateef]

Just wanted to share that we also use (multiple) ConcurrentHashMap to store common data that multiple sessions would need.

The ConcurrentHashMap objects are used as work queues for sessions,
or as a data context for FSM,
or to store for data that must be saved and resent on the next request.

In our case, the RESTful client is more complex than what a browser would ever do.

[John_Arrowwood1]

Thanks, I have coded up something and will test it thoroughly tomorrow.

[jarrowwx]

I wanted to capture the final solution, for posterity, in case other people are searching the group looking for a solution to a related problem. It’s not full featured, but it does the job for me:

`
import scala.collection._
import java.util.concurrent.ConcurrentHashMap
import scala.collection.JavaConverters._

class LookupCache {
val cache : concurrent.Map[String,String] = new ConcurrentHashMap() asScala
val locked : concurrent.Map[String,String] = new ConcurrentHashMap() asScala

// first thread to call lock(key) returns true, all subsequent ones will return false
def lock ( key: String, who: String ) : Boolean = locked.putIfAbsent(key, who ) == None

// only the thread that first called lock(key) can call put, and then only once
def put( key: String, value: String, who: String ) =
if ( locked.get( key ).get == who )
if ( cache.get( key ) == None )
cache.put( key, value )
else
throw new Exception( "You can’t put more than one value into the cache! " + key )
else
throw new Exception( “You have not locked '” + key + “’” )

// any thread can call get - will block until a non-null value is stored in the cache
// WARNING: if the thread that is holding the lock never puts a value, this thread will block forever
def get( key: String ) = {
if ( locked.get( key ) == None )
throw new Exception( “Must lock '” + key + “’ before you can get() it” )
var result : Option[String] = None
do {
result = cache.get( key )
if ( result == None ) Thread.sleep( 100 )
} while ( result == None )
result.get
}
}

`

And I use it like this:

`
object Login {
. . .
private val tokenCache = new LookupCache()

def process =
doIf( session => tokenCache.lock( session( USER_NAME ).as[String], session.userId ) ) {
// code that does the login and stores the token into TOKEN_VALUE
. . .
// save the token into the cache for next time
.exec( session => {
tokenCache.put(
session( USER_NAME ).as[String],
session( TOKEN_VALUE ).as[String],
session.userId
)
session
})
}
// pull the token out of the cache for use in subsequent tests
.exec( session => session.set( TOKEN_VALUE, tokenCache.get( session( USER_NAME ).as[String] ) ) )
}

. . .

val scn = scenario( MyTest )
.exec( Login.process )

`

Now, if two Gatling users try to log in as the same end-user, both will use the same token, and all is well. YAY!

[Excilys]

Nice!

[Priyanka]

Hi..thanks for the post..I have same requirement but the token gets expire for every one hour..I need to pass new token for every hour..could u pls suggest me the solution

[John_Arrowwood1]

What a client application is supposed to do is inspect the response code. If the response indicates that the token is expired, they should fire off a request to renew it, then re-submit the original request.

In order for you to do the same thing, you will want a single point of interface between your script and the service. All the service endpoints drive through a single function, which wraps the request and inspects the result for an expired token, and automatically renews it if it is expired. Then, it has to re-try the original request.

Or, you can do what I did: generate the tokens with a separate process, with long-lasting tokens (good for a year), and upload them directly to the database. That way your scripts can get the tokens from a feeder. It may not be exactly the way the clients will use the service, but it keeps things simple, and has next to no impact on the validity of the load test.

I chose that approach in part because of the difficulty of the other approach. You would have to write a wrapper function, which you would then have to inject into every service call in your script. I found that level of repetition offensive. But that’s me.

[Priyanka]

Thank u so much John…

[NewBee]

hi John,

I’m from Python background, Trying Gatling.

can u provide me a gist of the whole code base, which contains the Login Object and the Simulation class.
I’m unable to understand the concept of session. Believe looking at the code can help me.

My requirement is also similar. I have to get token, create user as pre-requisite. Then make sure user can do many activities.

Hence i had my setup like this:

setUp(GetAuthToken.authenticate.inject(atOnceUsers(1)).protocols(api_svc),
   CreateUser.source.inject(atOnceUsers(1)).protocols(api_svc),

   SendMessage.message.inject(atOnceUsers(1)).protocols(ingest_svc))

But i’m unable to pass the session variables from GetAuthToken scenario to CreateUser scenario.

Any help would be much appreciated.

Thanks in advance,
Ashok Kumar C