Alternatives for removed Gatling report files (3.13.5)

[sbatteux]

Gatling version: 3.13.5
Gatling flavor: java kotlin scala javascript typescript
Gatling build tool: maven gradle sbt bundle npm

I made sure I’ve update my Gatling version to the latest release
I read the guidelines and how to ask a question topics.
I provided a SSCCE (or at least, all information to help the community understand my topic)

Hello,

Since upgrading to version 3.13.5, several reporting files are no longer generated:

• global_stats.json: Required for the Gatling Jenkins plugin

• assertions.xml: Required for the JUnit Jenkins plugin and considered a de facto standard format for test results

• stats.json: Required for internal integration

These files were removed in the following commit: https://github.com/gatling/gatling/commit/711b8d4e7ac7aaa8d3173b2d77fb5e9c7843695a

From these different sources, I understand that no generated file is considered as “public interface” of the Gatling SDK and therefore can be removed, fully or partially, without notice:

• stats.json, global_stats.json, assertions.xml:

  • Removal PR description: https://github.com/gatling/gatling/commit/711b8d4e7ac7aaa8d3173b2d77fb5e9c7843695a

  • People are stuck:

    • https://community.gatling.io/t/how-to-extract-stats-from-the-report-and-persist-then-in-a-csv-or-database/9742
    • https://community.gatling.io/t/generate-gatling-report-from-multiple-simulation-log-files-with-version-3-13-5/9737
    • https://github.com/gatling/gatling/issues/4630

• assertions.json:

  • Removed since several releases: https://github.com/gatling/gatling/commit/42eedd15669d4a40a05f92e508ffc15b122a0faa

  • Impacting several users, raising multiple bugs on Gatling’s Github project. Issues #4630, #4522, #4506 and #4500

• simulation.log

  • As stated in documentation: https://docs.gatling.io/tutorials/faq/#logfile

  • Regularly reminded in topics:

    • https://community.gatling.io/t/generate-gatling-report-from-multiple-simulation-log-files-with-version-3-13-5/9737

    • https://community.gatling.io/t/the-simulation-log-file-is-not-utf-8-encoded/9426

My question is:

What are the alternatives for open-source users to retrieve the same information and regenerate these files within their own simulation or integration code? Specifically:

• Is there a documented public file or data class we can rely on?

• Is there a hook or listener that provides callbacks?

• Can we implement a custom DataWriter to collect and export the required information?

Thank you for your guidance on this. This is currently a blocker for our upgrade to the latest version.

[slandelle]

Hello,

TL;DR: if you’re building on internal components, please know that you’re doing so at your own risk—and outside the boundaries of what we support.

We understand this may disrupt workflows that depended on those files, but since they were never considered stable nor public, we can’t offer support or alternatives for them in Gatling open-source.

Note: the free Jenkins plugin was donated to the Jenkins community and AFAIK, a new maintainer has taken over and released a new version that’s compatible with latest Gatling versions.

Thanks for your understanding

[sbatteux]

Thanks for your response.

I don’t fully agree with the fact that these files were never considered stable nor public as they had been around and used by the Jenkins plugin for a decade. Also the assertions files were still mentioned in the 3.10.3 documentation (Dec 2023): https://github.com/gatling/gatling/blob/v3.10.3/src/docs/content/reference/current/core/assertions/index.md#reports

Regarding the Jenkins plugin, Gatling 3.14 support was rollbacked just a week after its introduction because the plugin was forced to parse the report.html file — a solution ultimately deemed unviable. I believe we can all agree on that point.
https://github.com/jenkinsci/gatling-plugin/commit/789314289eb23e7cb7cdd1d78471366c0aac0d97

I understand that at the moment, Gatling doesn’t provide any way to access/collect information that was available in the removed files in order to generate them on user side. However, if no alternative is implemented, it would leave Jenkins and similar integrations without a viable path forward. Considering the number of user requests, the broad impact, and the relatively small size of the pull request that removed this functionality, restoring it would clearly serve the best interests of the community.

Thank you for considering this request, and I look forward to your feedback.

[slandelle]

Regarding the Jenkins plugin, Gatling 3.14 support was rollbacked just a week after its introduction because the plugin was forced to parse the report.html file

No, that’s not the reason why this commit was reverted.

An older version of this plugin would let you directly display the Gatling HTML reports in the Jenkins UI. This feature was considered a vulnerability by the Jenkins security team and had to be removed. Instead it would let you download the reports.

This commit tried to re-introduce this feature, the Jenkins security team found out and required the new maintainer to revert it.

Anyway, we’ve added an entry regarding our position on this matter in the FAQ. This position won’t change.

Regards,

Note: I had to edit your post. Kind reminder: as per this forum rules, advertising competitors is prohibited.