How to transform response body before extraction with JSONPath?

Gatling (Open-Source)
1

I’ve run into a little data transformation problem, perhaps someone could point me in the right direction?

I need to call a REST API that returns JSON, extract an item from the result, and save its ID for future reference. Normally, this would be easily done with .check, .jsonPath and .saveAs but there’s a snag: the JSON is preceded by an AngularJS JSON vulnerability protection token )]}’, so it is not actually valid JSON.

Let’s say my API is /api/gizmos/ and it returns the following:

)]}’,
{“gizmos”:[{“id”: “ABC”},{“id”: “XYZ”}]}

I cannot use the following because the response body is not valid JSON:

http(“all gizmos”)
.get("/api/gizmos/")
.headers(my_custom_headers)
.check(jsonPath("$.gizmos[0].id").saveAs(“first_id”))

Gatling will then complain as follows:

all gizmos: KO jsonPath($.gizmos[0].id).find(0).exists failed, could not prepare: Could not parse response into a JSON object: Unable to determine the current character, it is not a string, number, array, or object

The current character read is ‘)’ with an int value of 41
Unable to determine the current character, it is not a string, number, array, or object
line number 1
index number 0
)]}’,
^

I’ve tried adding a .transformResponse to delete the token before doing the check, but I get the same error:

http(“all gizmos”)
.get("/api/gizmos/")
.headers(my_custom_headers)
.transformResponse {
case response if response.isReceived =>
new ResponseWrapper(response) {
override val body = StringResponseBody(response.body.string.replace(")]}’,", “”), UTF_8)
}
}
.check(jsonPath("$.gizmos[0].id").saveAs(“first_id”))

I cannot do .check(bodyString.transform(…) because jsonPath cannot be used then (it can only be used as a starting point)

I’m using Gatling 2.1.5. What am I doing wrong?

2

Using a transformResponse should work, are you sure you properly tested?

Note that you should:

  • use stripPrefix instead of replace
  • use response.charset instead of hardcoding UTF_8
3

The flow doesn’t seem to get into the body of the transformResponse. I experimented with disabling the guard condition after the case response and adding some println’s but to no avail. None of these println’s show up in the output:

  .transformResponse {
  case response /*if response.isReceived*/ =>
    println("*** creating ResponseWrapper")
    new ResponseWrapper(response) {
      val stripped = response.body.string.stripPrefix(")]}',")
      println("*** stripped=" + stripped)
      override val body = StringResponseBody(stripped, response.charset)
    }
  case _ =>
    println("*** something else happened")
    null
}

The following println just before calling the API does show up in the console, so the println itself works fine and assures me that my changes indeed affect the log output (i.e. I’m not looking at stale output).

.exec(session => {
  println("*** going to call API!")
  session
})
4

I have no idea what happens on your side. The following example works just fine for me with Gatling 2.1.5:

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.jdbc.Predef._
import io.gatling.http.response.ResponseWrapper
import io.gatling.http.response.StringResponseBody

class PerformanceTest extends Simulation {

val httpProtocol = http
.baseURL(“http://gatling.io/”)

val scn = scenario(“Basic”)
.exec(http(“Home”).get("/")
.transformResponse {
case response if response.isReceived =>
println("*** creating ResponseWrapper")
new ResponseWrapper(response) {
val prefixed = “FOOBAR” + response.body.string
println("*** prefixed=" + prefixed)
override val body = StringResponseBody(prefixed, response.charset)
}
}.check(substring(“FOOBAR”).count.is(1)))

setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}

Please provide a reproducer.

5

Found the culprit! My API was being called within a .resources() and then it apparently does not do the transformResponse correctly. Which is odd, because there are no compile errors that indicate I shouldn’t be calling it there. When I moved the API call outside the .resources it is processed correctly as expected.

This code reproduces the problem:

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import io.gatling.http.response.ResponseWrapper
import io.gatling.http.response.StringResponseBody

class PerformanceTest extends Simulation {

  val httpProtocol = http
    .baseURL("http://whois.icann.org/en")

  val scn = scenario("Basic")
    .exec(
      http("Regular").get("/about-whois")
        .transformResponse {
        case response if response.isReceived =>
          println("*** creating regular ResponseWrapper")
          new ResponseWrapper(response) {
            val prefixed = "FOOBAR" + response.body.string
            println("*** prefixed (regular)=" + prefixed.substring(0, 100))
            override val body = StringResponseBody(prefixed, response.charset)
          }
      }.check(substring("FOOBAR").count.is(1))
        .resources(
          http("Resources").get("/policies")
            .transformResponse {
            case response if response.isReceived =>
              println("*** creating ResponseWrapper inside resources")
              new ResponseWrapper(response) {
                val prefixed = "BAZQUX" + response.body.string
                println("*** prefixed inside resources=" + prefixed.substring(0, 100))
                override val body = StringResponseBody(prefixed, response.charset)
              }
          }.check(substring("BAZQUX").count.is(1)))
    )

  setUp(scn.inject(atOnceUsers(1))).protocols(httpProtocol)
}

Because gatling.io is a single-page app, fetching a different page was apparently hitting the cache, which I didn’t want, so I used whois.icann.org instead.
Relevant part of the console output:
2015-07-17 10:55:09,250 INFO Sending request=Regular uri=http://whois.icann.org/en/about-whois: scenario=Basic, userId=4170040969285647757-0

6

Fixed, thanks for reporting!
https://github.com/gatling/gatling/issues/2772

7

Willem Alexender,

Did this ever work for you? And if so, which Gatling/jdk version?

See my thread here; I’m trying to do the same as you, but there is not way I get it to work. On Gatling 2.2.2 I get a compilation error; on Gatling 2.1.7 the same code compiles, but at runtime the body isn’t modified.

I would appreciate it very much if you could give your thoughts about this.

Thanks,
Nol

8

Hello Nol,

I think I haven’t retested it with the fixed version. I checked my e-mails from that time; when I discovered that my problem only occurred within .resources, I moved the calls outside and then everything worked fine. After those particular Gatling tests were successfully completed, I didn’t follow up on this issue anymore.

Kind regards,

Willem