Unzipping json file for feeders

I have seen examples of csv and this example works well with csv files. But I have a situation where i have huge json file, so i am zipping and trying to unzip the same to apply as feeder and get the object count as well. But this seems to be consistently failing.
I have researched on google, stackoverflow and other forums and even on this doc https://gatling.io/docs/gatling/reference/current/core/session/feeder/ , it doesn’t mention if unzip can be applied to json.

Has anyone come across this or do gatling contributors have suggestion?

    val jsonObj0 = jsonFile("supplyEvents0.json.zip").unzip
    val size0 = jsonObj0.readRecords.size

Error:

15:36:43.615 [ERROR] i.g.a.Gatling$ - Run crashed
com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0x8f

Also note there is not issue with this json file… its absolutely working fine when not zipped. The only reason I am zipping is because of its size as it causes failure while running from frontline enterprise tool.

Gatling version: 3.6.1

Do you getting any error message ?

Sorry just realized i had missed providing the error… just updated the post.
And yes i am getting parsing error and thats coming from line of ‘unzip’

Without zipping everything is working correct ?
How you zip file ?

yes without zipping everything works fine.

I am using mac to run local development.
First i used right click to zip it. And it gave the error.

Then I tried with zip abc.json.zip abc.json and this way also failed.
Now i am thinking to find a way to zip it keep that utf encoding in mind… but still researching

Try jar cf file.json.zip file.json (don’t remember the actual way to use jar to zip unzip files, been a long time). Usually this way helps if all fails with Java.

In my opinion there is a missing implementation of unzip for jsonFile - I make some tests for that, for CSV zipped in the same way everything is working.
So feeder try to parse unzipped file so is getting parse error.
@sbrevet or @slandelle can you confirm ? :slight_smile:

I thought there was issue with encoding, but i checked…

file -I sample.json
sample.json: application/json; charset=us-ascii

And on side, i also freshly created a simple json but still same error.

16:19:29.719 [ERROR] i.g.a.Gatling$ - Run crashed
com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0xb6

So I guess you may be correct this implementation is not available in gatling @GeMi

I confirm it’s a bug.
I’ve just pushed the fix that will be shipped in upcoming 3.8.3: Core: jsonFile and jsonUrl feeders should support unzip · Issue #4292 · gatling/gatling · GitHub

1 Like

Thanks Stéphane for such a quick turn around time. Since I don’t have eyes on the timeline view of your builds and releases, can you please share a tentative date when you thinking to release 3.8.3? It will help me plan with my team.

Maybe tomorrow if things go as expected.

1 Like