Looping and saving values from a json response and reusing the values in next request

fugmag · November 21, 2014, 9:20am

Hello,

I have a case where I my JSON response body consists of the following:

`

{
“resultCode”: “SUCCESS”,
“errorCode”: null,
“errorMessage”: null,
“membershipCoupons”: [
{
“membershipId”: “136134567”,
“coupons”: [
{
“offerId”: “000C2977456”,
“description”: "PRODUCT_1 ",
“graphics”: [
{
“width”: 400,
“height”: 200,
“quality”: “Low”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C2977456Low.jpg”
},
{
“width”: 900,
“height”: 600,
“quality”: “Hight”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C2977456High.jpg”
}
],
“endTime”: 1417305600000,
“activationDate”: 1416244318947,
“redemption”: null
}
]
},
{
“membershipId”: “810456459”,
“coupons”: [
{
“offerId”: “000C297765971E23432”,
“description”: "PRODUCT_2 ",
“graphics”: [
{
“width”: 400,
“height”: 200,
“quality”: “Low”,
“url”: “\n http://www.maersk.dk:8080/02_201442_005/000C297765971E23432Low.jpg”
},
{
“width”: 900,
“height”: 600,
“quality”: “Hight”,
“url”: “\n http://www.maersk.dk:8080/02_201442_005/000C297765971E23432High.jpg”
}
],
“endTime”: 1417305600000,
“activationDate”: null,
“redemption”: null
},
{
“offerId”: “000C29776FR5456065F7”,
“description”: "PRODUCT_3 ",
“graphics”: [
{
“width”: 400,
“height”: 200,
“quality”: “Low”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C29776FR5456065F7Low.jpg”
},
{
“width”: 900,
“height”: 600,
“quality”: “Hight”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C29776FR5456065F7High.jpg”
}
],
“endTime”: 1417305600000,
“activationDate”: null,
“redemption”: null

}
]
}
]
}

`

I want to collect all ‘offerId’s’ from the getCouponList and use them in the next request (activateCoupon). In this concrete example that is two offerId’s.

This code is now working for one (the first ‘activationDate=null’ it finds), but I want to expand this functionality to save all offerId’s where activationDate=null at once.

I guess in pseudo-code this will be:

getCouponlist() and while there are coupons left in the json response with activationDate=null - save them to be reused
Use the saved list form 1) and repeat the activateCoupon() until the list is empty
Maybe something like:

.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").saveAs(“offerId”))

while (offerId != null) repeat the saving of offerIds.

Here is the simulation code:

`

package dk.maersk

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._

class AppSimulation extends Simulation {

val httpConf = http
.baseURL(“https://site.maersk.dk”)
.acceptEncodingHeader(“gzip,deflate”)
.headers(Map(“Content-Type” → “application/json”, “charset” → “UTF-8”, “User-Agent” → “Android(4.4)/Maersk(0.4)/0.1”))

val scn = scenario(“Scenario”)

.feed(csv(“memberInfo.csv”).random)

.exec(
http(“getProfile”)
.get("/user/profile")
.header( “X-Token”, “${memberID}” )
.check(status.is(200))
.check(jsonPath("$.resultCode").is(“SUCCESS”))
.check(jsonPath("$.profile.memberships[0].number").is("${memberID}"))
)

.exec(
http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “${memberID}”)
.check(status.is(200))
.check(jsonPath("$.resultCode").is(“SUCCESS”))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").saveAs(“offerId”))
)

.exec(
http(“activateCoupon”)
.post("/coupon/activate")
.header(“X-Token”, “${memberID}”)
.body(ELFileBody(“activateCoupons.json”)).asJSON
.check(status.is(200))
.check(jsonPath("$.resultCode").is(“SUCCESS”))
)

setUp(scn.inject(rampUsersPerSec(1) to(100) during (5 minutes))).maxDuration(7 minutes).protocols(httpConf)
}

`

My request body looks like this:

`
{“offerId”: “${offerId}”, “membershipId”: “${memberID}”, “osName”: “Android”, “osVersion”: “4.4”}

`

and I have a .csv file with memberId’s.

So how would i traverse the json response and saving offerIds for reuse in the subsequent request as long as there is elements in the json body with activtatioDate = null ?

Stig_Brautaset · November 21, 2014, 6:39pm

Hi Magnus,

Here’s what we do, for a similar situation (please forgive the lisp-inspired indentation):

val scn = scenario("Seekrit Scenario 1")
  .during(duration.minutes) {
    feed(csv(s"data/urls.csv").random)
      .exec(_.set("itemList", Seq.empty[Int]))
      .exec(http("Get list of items")
        .get("${url}")
        .check(jsonPath("$.data[*]").ofType[Int].findAll.transform(_.take(pageSize)).optional.saveAs("itemList")))
      .foreach ("${itemList}", "item") {
        exec(http("Get individual item")
          .get("/item/${item}")
          .check(jsonPath("$[0].id").is("${item}")))}}

Hope this helps!

The reason for setting the empty itemList on start is to avoid an error if the .optional part of the save fails.

Stig

jarrowwx · November 21, 2014, 6:50pm

The key part to notice is “.findAll” which will select multiple values and store them in a Vector.

fugmag · November 21, 2014, 8:30pm

So I need to use ‘findAll’ and put all the values into a Vector.
The problem I then observed is that I use a request-body in my next request and I need to refer each item (from the Vector) in the request-body looking like this:

`
{“offerId”: “${offerId}”, “membershipId”: “${memberID}”, “osName”: “Android”, “osVersion”: “4.4”}

`

How can i do that?

My updated simulation code (based on your tips) will probably be like the following:

`

class testSimulation extends Simulation {

val httpConf = http
.baseURL(“https://site.maersk.dk”)
.acceptEncodingHeader(“gzip,deflate”)
.headers(Map(“Content-Type” → “application/json”, “charset” → “UTF-8”, “User-Agent” → “Android(4.4)/Maersk(0.4)/0.1”))

val scn = scenario(“Scenario”)
.feed(csv(“memberInfo.csv”).random)
.exec(_.set(“itemList”, Seq.empty[Int]))
.exec(http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “1361300123”)
.check(status.is(200))
.check(jsonPath("$.resultCode").is(“SUCCESS”))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”))

.foreach("${itemList}", “item”)
{
exec(http(“activateCoupon”)
.post("/coupon/activate")
.header(“X-Token”, “1361300123”)
.body(ELFileBody(“activateCoupons.json”)).asJSON
.check(jsonPath("$.resultCode").is(“SUCCESS”))
}
)
)
setUp(scn.inject(rampUsersPerSec(1) to (50) during(1 minutes)).protocols(httpConf))
}

`

Thanks for great help so far guys!

Magnus

jarrowwx · November 22, 2014, 2:21am

I’m a bit hazy on the question. But I will say this: You do not use “item” in your foreach loop. The only thing that will vary in your session while you loop will be that, so you will need to make sure that your ELFileBody is referencing it. If you need to get more than one value, it is a little more work. Or, if you need to compose one request from all of the values in the vector, you need to do some Scala logic.

fugmag · November 25, 2014, 2:28pm

Ok, I decided to exclude the use of request body an just use a regular StringBody like this:

`

.exec(http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “${memberID}”)
.check(status.is(200))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”)))

.foreach("${itemList}", “item”){
exec(http(“activateCoupon”)
.post("/coupon/activate")
.header(“X-Token”, “${memberID}”)
.body(StringBody("""{“offerId”: “${item}”, “membershipId”: “${memberID}”, “osName”: “Android”, “osVersion”: “4.4”}""")).asJSON
.check(jsonPath("$.resultCode").is(“SUCCESS”)))
}

`

Would this work as is? Can I just say ${item} and just refer to the next item in the Vector?

Excilys · November 25, 2014, 2:42pm

Absolutely

fugmag · November 25, 2014, 2:48pm

Is there an easy way to print the Vector to console?
As in using something like this:

`

.exec(session2 => {
println((session2(“item”).as[String]))
session2

`

Excilys · November 25, 2014, 3:39pm

Did you give it a try?

fugmag · November 25, 2014, 7:47pm

It worked, cheers!

Excilys · November 25, 2014, 7:49pm

Of course it did!

fugmag · November 25, 2014, 8:18pm

fugmag · November 25, 2014, 8:34pm

Hello,

Using the ‘optional’ like this:

`
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").optional.saveAs(“itemList”)))

`

gives me the error:

21:29:39.525 [ERROR] i.g.c.s.LoopBlock$ - Condition evaluation crashed with message 'Can't cast value 000C297765971EE495BE0BA921A2E5F7 of type class java.lang.String into interface scala.collection.Seq', exiting loop

But if I say

`
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”)))

`

It picks up all that values with ‘activationDate==null’ as expected.

Why should I use:

`
.optional.saveAs(“itemList”)))

`

?

fugmag · November 25, 2014, 9:03pm

And, there could be cases where my JSON response actually has no items with activationDate=null i.e. like this:

`

{
“resultCode”: “SUCCESS”,
“errorCode”: null,
“errorMessage”: null,
“membershipCoupons”: [
{
“membershipId”: “136134567”,
“coupons”: [
{
“offerId”: “000C2977456”,
“description”: "PRODUCT_1 ",
“graphics”: [
{
“width”: 400,
“height”: 200,
“quality”: “Low”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C2977456Low.jpg”
},
{
“width”: 900,
“height”: 600,
“quality”: “Hight”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C2977456High.jpg”
}
],
“endTime”: 1417305600000,
“activationDate”: 1416244318947,
“redemption”: null
}
]
},
{
“membershipId”: “810456459”,
“coupons”: [
{
“offerId”: “000C297765971E23432”,
“description”: "PRODUCT_2 ",
“graphics”: [
{
“width”: 400,
“height”: 200,
“quality”: “Low”,
“url”: “\n http://www.maersk.dk:8080/02_201442_005/000C297765971E23432Low.jpg”
},
{
“width”: 900,
“height”: 600,
“quality”: “Hight”,
“url”: “\n http://www.maersk.dk:8080/02_201442_005/000C297765971E23432High.jpg”
}
],
“endTime”: 1417305600000,
“activationDate”: 1416244318947,
“redemption”: null
},
{
“offerId”: “000C29776FR5456065F7”,
“description”: "PRODUCT_3 ",
“graphics”: [
{
“width”: 400,
“height”: 200,
“quality”: “Low”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C29776FR5456065F7Low.jpg”
},
{
“width”: 900,
“height”: 600,
“quality”: “Hight”,
“url”: “\n http://www.mearsk.dk:8080/02_201442_005/000C29776FR5456065F7High.jpg”
}
],
“endTime”: 1417305600000,
“activationDate”: 1416244318947,
“redemption”: null

}
]
}
]
}

`

If I then run my simulation with this:

`

.exec(http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “${memberID}”)
.check(status.is(200))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”)))

.foreach("${itemList}", “item”){
exec(http(“activateCoupon”)
.post("/coupon/activate")
.header(“X-Token”, “${memberID}”)
.body(StringBody("""{“offerId”: “${item}”, “membershipId”: “${memberID}”, “osName”: “Android”, “osVersion”: “4.4”}""")).asJSON
.check(jsonPath("$.resultCode").is(“SUCCESS”)))

…

`

I get the error:

`
jsonPath($.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId).findAll.exists, found nothing

`

Is there wa way to not get his error? I want to check that the .check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”)))

in:

`

.exec(http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “${memberID}”)
.check(status.is(200))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”)))

`

does not try to save the values it there are no activationDate==null like setting som condition before the saveAs, maybe in pseudocode:

if .check(jsonPath("$.membershipCoupons[].coupons[?(@.activationDate==null)].offerId") is not null
then
.check(jsonPath("$.membershipCoupons[].coupons[?(@.activationDate==null)].offerId").findAll.saveAs(“itemList”)))
else
skip this exec (without error)

Excilys · November 25, 2014, 10:28pm

See note: http://gatling.io/docs/2.0.3/http/http_check.html#extracting

fugmag · November 25, 2014, 10:28pm

It seems this works:

`
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.optional.saveAs(“itemList”)))

`

At least I do not get any errors in my report now, but I have ‘used-up’ all my testdata so I cannot confirm for sure until I have tested against json responses with activationDate==null and some json responses with all ‘activateionDate’ s set to a value.

I should search for the forum more before asking questions next time

fugmag · November 25, 2014, 10:37pm

But now the problem is that it continuous to run the next .exec even if I say: exitHereIfFailed

I might need a if-check to check if the item list is notEmpty before ‘exec(http(“activateCoupon”)…’ else I would not perform the ‘activate’ Any tips?

`

.exec(http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “${memberID}”)
.check(status.is(200))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.optional.saveAs(“itemList”))).exitHereIfFailed

.foreach("${itemList}", “item”){
exec(http(“activateCoupon”)
.post("/coupon/activate")
.header(“X-Token”, “${memberID}”)
.body(StringBody("""{“offerId”: “${item}”, “membershipId”: “${memberID}”, “osName”: “Android”, “osVersion”: “4.4”}""")).asJSON
.check(jsonPath("$.resultCode").is(“SUCCESS”)))

`

jarrowwx · November 26, 2014, 4:39am

You’ve got it. By using optional, it doesn’t fail. Before you do the request, set the list to empty. Then do the request/check, then do an “if” to make sure there is work to do.

fugmag · November 26, 2014, 10:45am

Setting up an if-check is obviously not that easy for me

I have tried this:

`

.exec(http(“getCouponList”)
.get("/coupon/all")
.header(“X-Token”, “${memberID}”)
.check(status.is(200))
.check(jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll.optional.saveAs(“itemList”))).exitHereIfFailed

if("${itemList}".isEmpty).exitHereIfFailed{
else

.foreach("${itemList}", “item”){
exec(http(“activateCoupon”)
.post("/coupon/activate")
.header(“X-Token”, “${memberID}”)
.body(StringBody("""{“offerId”: “${item}”, “membershipId”: “${memberID}”, “osName”: “Android”, “osVersion”: “4.4”}""")).asJSON
.check(jsonPath("$.resultCode").is(“SUCCESS”)))
}
}
}

`

But it fails.

Any tips?

Excilys · November 26, 2014, 12:29pm

You can use transformOption so you can transform the extracted value before applying the validation step.

jsonPath("$.membershipCoupons[*].coupons[?(@.activationDate==null)].offerId").findAll
.transformOption {
case None => Some(Nil) // couldn’t extract anything, replacing with an empty list
case someNEL => someNEL
}.saveAs(“itemList”)

Topic		Replies	Views
Saving and reusing JSON response Gatling (Open-Source)	5	119	October 14, 2014
save JSON response as key value pair Gatling (Open-Source)	0	95	August 4, 2017
Saving bigger chunks of a json response to be used in subsequent requests Gatling (Open-Source)	2	706	December 23, 2022
retrieve a value from json response and add it to the list, iterate through list Gatling (Open-Source)	3	339	March 26, 2021
picking just the first occurence in a json response using .check Gatling (Open-Source)	0	106	February 11, 2015

Looping and saving values from a json response and reusing the values in next request

Related Topics