Hi community,
We have developed a Gatling loadtest testing an Application Lifecycle Management (ALM) tool. There are four scenarios:
- requirements-engineer
- lead-engineer
- test-engineer
*business admin.
Based on quality criteria we used the assertions method to limit the response times for each step of the archetypes above and also max. response times for each in sum.
The setup is listed below:
setUp(
preparation.inject(
atOnceUsers(if (withPreparation) 1 else 0)
).protocols(httpProtocolAdmin)
.andThen(
inject(businessAdministrator,
rampNumberOfBusinessAdministratorsFrom,
toNumberOfBusinessAdministrators)
.protocols(httpProtocolAdmin),
inject(requirementsEngineer,
rampNumberOfRequirementsEngineersFrom,
toNumberOfRequirementsEngineers)
.protocols(httpProtocolRequirementEngineer),
inject(leadEngineer,
rampNumberOfLeadEngineersFrom,
toNumberOfLeadEngineers)
.protocols(httpProtocolLeadEngineer),
inject(testEngineer,
rampNumberOfTestEngineersFrom,
toNumberOfTestEngineers)
.protocols(httpProtocolTestEngineer)
)
).assertions(myAssertions.defineAssertions())
package com.vwgroup.codebeamer.cbloadtests.assertions
import io.gatling.core.Predef.{global, _}
import com.vwgroup.codebeamer.cbloadtests.utils.Configuration._
class Assertions {
def defineAssertions(): Seq[Assertion] = {
Seq(
global.responseTime.max.lt(70000),
details("Requirements Engineer").responseTime.max.lt(70000), // 1,1 fache max // sum max. response time
details("Requirements Engineer" / "login").responseTime.max.lt(3000),
details("Requirements Engineer" / "createTrackerItem").responseTime.max.lt(1000),
details("Requirements Engineer" / "changeTrackerIssues").responseTime.max.lt(1000),
details("Requirements Engineer" / "excelRoundTrip").responseTime.max.lt(30000),
details("Requirements Engineer" / "exportReqIf").responseTime.max.lt(30000),
details("Business Administrators").responseTime.max.lt(100000),
details("Business Administrators" / "login").responseTime.max.lt(3000),
details("Business Administrators" / "exportReqIf").responseTime.max.lt(30000),
details("Business Administrators" / "createProjectViaWorkflow").responseTime.max.lt(60000), // Intland Default Project
details("Business Administrators" / "createUser").responseTime.max.lt(1000),
details("Business Administrators" / "editUser").responseTime.max.lt(1000),
details("Business Administrators" / "addEditGroup").responseTime.max.lt(1000),
details("Business Administrators" / "deleteUserGroup").responseTime.max.lt(1000),
details("Lead Engineer").responseTime.max.lt(25000),
details("Lead Engineer" / "login").responseTime.max.lt(3000),
details("Lead Engineer" / "copyFromOriginalToDuplicateTracker").responseTime.max.lt(5000),
details("Lead Engineer" / "changeTrackerIssues").responseTime.max.lt(1000),
details("Lead Engineer" / "createReview").responseTime.max.lt(5000), // for 50 items (30000ms/500items)
details("Lead Engineer" / "completeReview").responseTime.max.lt(5000), // for 50 items (30000ms/500items)
details("Lead Engineer" / "createTrackerItem").responseTime.max.lt(1000),
details("Lead Engineer" / "repeatCreateTrackerItem").responseTime.max.lt(2000),
details("Test Engineer").responseTime.max.lt(20000),
details("Test Engineer" / "login").responseTime.max.lt(3000),
details("Test Engineer" / "createTestCase").responseTime.max.lt(1000),
details("Test Engineer" / "createTestSet").responseTime.max.lt(1000),
details("Test Engineer" / "addCasesToTestSet").responseTime.max.lt(1000),
details("Test Engineer" / "createTestRun").responseTime.max.lt(5000), // for 50 items (30000ms/500items)
details("Test Engineer" / "updateTestRun").responseTime.max.lt(5000), // for 50 items (30000ms/500items)
global.successfulRequests.percent.gt(95),
global.failedRequests.percent.lte(2)
)
}
}
My question would be, if the defined assertions and the used methos is correct like that or if there is a way to optimize this or use it smarter. Else, for the archetypes as group, we assumed the cumulative maximum response times for tasks within the archetype and it is calculated as 1.1 times the anticipated maximum duration for these tasks.
Is there a way to integrate a formula into the code and what would be a realistic factor based on the response times we set for each step?
responseTimeLogin = testProp.getProperty("responseTimeLogin", "3000").toInt
responseTimeCreateTrackerItem = testProp.getProperty("responseTimeCreateTrackerItem", "1000").toInt
responseTimeChangeTrackerIssues = testProp.getProperty("responseTimeChangeTrackerIssues", "1000").toInt
responseTimeExcelRoundTrip = testProp.getProperty("responseTimeExcelRoundTrip", "30000").toInt
responseTimeExportReqIf = testProp.getProperty("responseTimeExportReqIf", "30000").toInt
responseTimeCreateProjectViaWorkflow = testProp.getProperty("responseTimeCreateProjectViaWorkflow", "60000").toInt
responseTimeCreateUser = testProp.getProperty("responseTimeCreateUser", "1000").toInt
responseTimeEditUser = testProp.getProperty("responseTimeEditUser", "1000").toInt
responseTimeAddEditGroup = testProp.getProperty("responseTimeAddEditGroup", "1000").toInt
responseTimeDeleteUserGroup = testProp.getProperty("responseTimeDeleteUserGroup", "1000").toInt
responseTimeCopyFromOriginalToDuplicateTracker = testProp.getProperty("responseTimeCopyFromOriginalToDuplicateTracker", "5000").toInt
responseTimeCreateReview = testProp.getProperty("responseTimeCreateReview", "5000").toInt
responseTimeCompleteReview = testProp.getProperty("responseTimeCompleteReview", "5000").toInt
responseTimeRepeatCreateTrackerItem = testProp.getProperty("responseTimeRepeatCreateTrackerItem", "20000").toInt
responseTimeCreateTestCase = testProp.getProperty("responseTimeCreateTestCase", "1000").toInt
responseTimeCreateTestSet = testProp.getProperty("responseTimeCreateTestSet", "1000").toInt
responseTimeAddCasesToTestSet = testProp.getProperty("responseTimeAddCasesToTestSet", "5000").toInt
responseTimeCreateTestRun = testProp.getProperty("responseTimeCreateTestRun", "5000").toInt
responseTimeUpdateTestRun = testProp.getProperty("responseTimeUpdateTestRun", "5000").toInt
// From Test
var responseTimeLogin: Int = 3000
var responseTimeCreateTrackerItem: Int = 1000
var responseTimeChangeTrackerIssues: Int = 1000
var responseTimeExcelRoundTrip: Int = 30000
var responseTimeExportReqIf: Int = 30000
var responseTimeCreateProjectViaWorkflow: Int = 60000
var responseTimeCreateUser: Int = 1000
var responseTimeEditUser: Int = 1000
var responseTimeAddEditGroup: Int = 1000
var responseTimeDeleteUserGroup: Int = 1000
var responseTimeCopyFromOriginalToDuplicateTracker: Int = 5000
var responseTimeCreateReview: Int = 5000
var responseTimeCompleteReview: Int = 5000
var responseTimeRepeatCreateTrackerItem: Int = 20000
var responseTimeCreateTestCase: Int = 1000
var responseTimeCreateTestSet: Int = 1000
var responseTimeAddCasesToTestSet: Int = 5000
var responseTimeCreateTestRun: Int = 5000
var responseTimeUpdateTestRun: Int = 5000
Also we added variables as e.g. responseTimeLogin = 3000ms and configured under testParameters.properties and configuration.scala. Instead of declaring individual integers for the assertions so changes can be made from there.
So all in all i am looking forward for suggestions for improvement.
Thank you in advance.