I am sure you might have experienced a situation where with daily new features being dropped for your system under test, your simulations would become out of sync i.e. some new API calls will be introduced or some older once are removed from transactions/user journeys.
I would like to hear form all of you on how do you/your team keep up with these frequent changes? Do you manually check each user journey frequently (multiple times a week) in browser/app and sync with your simulation or does dev teams communicate changes regularly or do you use any automated way to keep your simulations in sync with ever changing user journey flows of your web/mobile app?
Your suggestions and inputs will help us streamline process in our project as well, thanks in advance.
I would recommend running smoke tests (your load tests with just 1 virtual user) on your CI so that you quickly detect that tests haven’t been updated along with the API.
Smoke tests might still pass if newly added or removed APIs are not dependent on other calls… running gatling simulations with 1 user will not confirm that flow is not broken.
It depends if the change in the API flow is a breaking one or not. If the latter, which is probably expected when dealing with mobile apps where you want to retain compatibility with older app versions that weren’t updated, I guess you’re seeing the limit of having the tests implemented by a third party that doesn’t belong to the project team and has to deal with undocumented changes.
Maybe try to record a HAR and detected differences from there.