Gatling scala script causing heap memory issue with huge data processing

[Monika]

Gatling Scala script causing heap memory issue with huge data processing.

Please help me with solution

I tried with gatling version 3.7.6 to 3.9.2 still issue.

I have to process 300000 data set in stretch ( condition I cannot batch or split the load)
Project ( gradlew kotlin - gatling -scala)

in all the steps I am getting after increasing the heap size to 4GB

• “java.lang.OutOfMemoryError: Java heap space”

there are no memory leaks in the code, but still gatling could not process such huge data.

1. I am creating Json variable holding 300000 data set
2. I am providing that Json variable to request to process & provide a response
3. I am save a session variable to hold 300000 data set fetched from response
4. I am looping 300000 times ( single data processing API schema) modifying 300000 data individually
5. Saving each processed data in to list holding 300000 data.
6. Deleting 300000 data set passed to a bulk request.

Please find my code block

val  testCount = 300000
.
.
.
    .exec { session =>
      val testtypeids = Array("000000001", "000001", "000001101", "00000000001")
      val statusID = Array("000008", "0000058", "00000000-00019", "00000000-0009")
      val PerfIDs = session("tperfId").as[Vector[String]]
      val testData = (1 to testCount)
        .map { r =>
          Json.toJson(Map(
            "name" -> Json.toJson(s"Perftest${Random.alphanumeric.take(6).mkString}"),
            "displayName" -> Json.toJson(s"Perftest${Random.alphanumeric.take(6).mkString}"),
            "PerfIDs" -> Json.toJson(s"${PerfIDs(r % PerfIDs.length)}"),
            "testtypeids" -> Json.toJson(s"${Random.shuffle(testtypeids.toList).head}"),
            "statusId" -> Json.toJson(s"${Random.shuffle(statusID.toList).head}"),
            "excludedFromAutoHyperlinking" -> Json.toJson(true)
          ))
        }
      val datatestJson = Json.toJson(testData)
      session.set("datatestJson", datatestJson)
    }

    .exec(http("Add json data holding 300000 data set")
      .post("test/bulk")
      .body(StringBody(session => s"""${session("datatestJson").as[String]}""")).asJson
      .requestTimeout(2.hours)
      .check(jsonPath("$[*].id").findAll.saveAs("testIDs"))
      .check(status.is(201)))
    .pause(2.seconds)

    .foreach("#{testIDs}", "TestID") {
      exec(http("update 300000 data individually")
        .post("updating/testdata")
        .body(StringBody(session =>
          s"""{
             |  "id": "",
             |  "additionalTypeId": "000000001",
             |  "legs": [
             |    {
             |      "additionalTypeId2": "00000000112311",
             |      "testID": "${session("TestID").as[String]}"
             |    }
             |    ]
             |  }""".stripMargin)).asJson
        .check(jsonPath("$.id").saveAs("newtestID"))
        .check(status.is(201)))
    }
    .pause(2.seconds)

    .exec(http("delete 300000 data set")
      .delete("/tests/bulk")
      .header("Content-Type", "application/json")
      .body(StringBody(session => s"""[ "${session("newtestID").as[Vector[String]].mkString("\",\"")}" ]"""))
      .requestTimeout(2.hours)
      .check(status.is(204)))
    .pause(2.seconds)

Please help me with solutions without compromising on 300000 data set request & value saving from response in stretch.

[sbrevet]

Hi @Monika,

First, what your injection profile looks like? If you have hundred of users, each one will load the 300k data.
Did you try with only one user?

@Monika

after increasing the heap size to 4GB

How did you do that? (only to be sure you effectively increased the Gatling simulation heap size and not only the packager one)

Why do you use string interpolation when you only have a string value?

-      .body(StringBody(session => s"""${session("datatestJson").as[String]}""")).asJson
+     .body(StringBody(session => session("datatestJson").as[String])).asJson

-      .body(StringBody(session => s"""[ "${session("newtestID").as[Vector[String]].mkString("\",\"")}" ]"""))
+      .body(StringBody(session => session("newtestID").as[Vector[String]].mkString("[ \"", "\",\"", "\" ]")))

@Monika

• Saving each processed data in to list holding 300000 data.

You mean each 300000 save 300000 data?

Cheers!

[Monika]

@sbrevet Thank you for responding

1. User Injection profile - 1 user

2. I have increased the gatling heap memory in build.gradle.kts
   gatling {
   logHttp = io.gatling.gradle.LogHttp.FAILURES
   logLevel = “DEBUG”
   jvmArgs = listOf(“-Xmx8192m”)
   }

3. I user will create a set of 300K data set then upon the created data set modifying it one by one by loop
   and then deleting 300K data set passing value to one API request.

[sbrevet]

Your code feels weird:

• newtestID should be a single value (the latest) as you override it for each update but you use it as a Vector[String] (that should fail)
• requestTimeout(2.hours) => is there any traffic on the line to keep the connection alive? that will be stored by gatling to be able to parse the whole body?

At what time (in which request) does the error happen?
Why putting the whole body “datatestJson” in session (you should be able to generate it in the StringBody function)?
At least, clean the session to avoid remember this huge payload

Please provide a full SCCE to be able to fully reproduce the issue on our side (and be able to analyze it)

Cheers!

[shoaib42]

Can you give a try with feeders, I think that way you won’t be hogging the memory.

As an example

def foobar : Map[String, Any] = {
  //return a map here, do you random picking
  Map("name" -> "batman", "PerfIDs" -> "#{tperfId.random()}")
}
val feeder = Iterator.continually(Map("bodymap" -> foobar()))

val myHttpCall = http("foo").post("/somewhere").body(StringBody("#{bodymap.jsonStringify()}").asJson

[slandelle]

Just sharing some insights, not providing a solution.

Here, memory usage is:

• the huge datatestJson input dataset (can’t say the type, don’t know what Json.toJson is)
• the huge String version from session => s"""${session("datatestJson").as[String]}"""
• the humongous response to the POST request, both bytes and JSON AST used underneath by the JsonPath check
• the humongous StringBody of the bulk DELETE request, once it’s fixed
• all the IO buffers required for reading and writing these payload on the socket

So, huge memory usage indeed.

Also, setting the Xmx to 8G doesn’t mean that the JVM has sufficient space to allocated this heap.

[Monika]

Thank you for the suggestions and solutions.
with the provided guidance I have modified the script & memory issue is solved.

val  testCount = 3000
.
.
.
.repeat(100) {
exec { session =>
      val testtypeids = Array("000000001", "000001", "000001101", "00000000001")
      val statusID = Array("000008", "0000058", "00000000-00019", "00000000-0009")
      val PerfIDs = session("tperfId").as[Vector[String]]
      val testData = (1 to testCount)
        .map { r =>
          Json.toJson(Map(
            "name" -> Json.toJson(s"Perftest${Random.alphanumeric.take(6).mkString}"),
            "displayName" -> Json.toJson(s"Perftest${Random.alphanumeric.take(6).mkString}"),
            "PerfIDs" -> Json.toJson(s"${PerfIDs(r % PerfIDs.length)}"),
            "testtypeids" -> Json.toJson(s"${Random.shuffle(testtypeids.toList).head}"),
            "statusId" -> Json.toJson(s"${Random.shuffle(statusID.toList).head}"),
            "excludedFromAutoHyperlinking" -> Json.toJson(true)
          ))
        }
      val datatestJson = Json.toJson(testData)
      session.set("datatestJson", datatestJson)
    }

    .exec(http("Add json data holding 300000 data set")
      .post("test/bulk")
      .body(StringBody(session => s"""${session("datatestJson").as[String]}""")).asJson
      .requestTimeout(2.hours)
      .check(jsonPath("$[*].id").findAll.saveAs("testIDs"))
      .check(status.is(201)))
    .pause(2.seconds)

    .foreach("#{testIDs}", "TestID") {
      exec(http("update 300000 data individually")
        .post("updating/testdata")
        .body(StringBody(session =>
          s"""{
             |  "id": "",
             |  "additionalTypeId": "000000001",
             |  "legs": [
             |    {
             |      "additionalTypeId2": "00000000112311",
             |      "testID": "${session("TestID").as[String]}"
             |    }
             |    ]
             |  }""".stripMargin)).asJson
        .check(jsonPath("$.id").saveAs("newtestID"))
        .check(status.is(201)))
    }
    .exec(session =>session.remove("testIDs"))
    .exec(session =>session.remove("datatestJson"))
}


// below segment can be ignored, I am deleting the entire web directory with a single API call ( measuring the time how long it takes to delete 300000 data sets)

    .exec(http("delete 300000 data set")
      .delete("/tests/bulk")
      .header("Content-Type", "application/json")
      .body(StringBody(session => s"""[ "${session("newtestID").as[String]}" ]"""))
      .requestTimeout(2.hours)
      .check(status.is(204)))
    .pause(2.seconds)