To give a concrete example of what I want to do, I need to test an authenticating proxy server in each of its possible deployment configurations.
The test includes a number of test parameters. Which deployment configuration of the we are hitting is one (ECS, EC2/Linux, EC2/Windows, and others), but there are also parameters that control the behavior of the dummy app that the proxy protects, like how big of a response to provide, or how much latency to deliberately inject. And there are also parameters like which user source to use, and whether to test Basic Auth or interactive login flows.
I realize I could just create separate jobs for each of these configurations, but then I have to do that for every test, and every such parameter, which results in a combinatorial explosion of pre-configured jobs. The way things are right now, if I went that way I would have to create 2x2x5x9x9 = 1,620 jobs. No thanks!
If it were a Jenkins job, I would setup the variables as parameters that lets me choose from the valid options, with an intelligently chosen default value. I would represent all of the above as either a single job, or at most, 4 jobs, where the default parameter values capture the most important or common test configuration.
Does FrontLine support such a feature? I didn’t see anything that suggested it in the docs, but I may have just missed it.