Selenium - Browser Website performance testing disadvantages

Gatling (Open-Source)
1

Hi,

This question has been asked multiple times and I am aware that the answer is “No”.
I just want to know more details about the disadvantages when trying to simulate Browser Web performance testing. There were not much solid answers out there.
Selenium, for example, could execute the test, record execution time, so how/why it is not capable with Gatling?

Thanks for reading.

2

Hey there,

Happy to help, they are really different sides of testing.

Gatling is highly scalable and can simulate a large number of virtual users concurrently, this is its primary purpose. Our tool focuses on performance testing from the server side (response times for requests, throughput and error ratios plus additional metrics on Gatling Enterprise including TLS handshake time, DNS resolutions, etc.) and does not have built-in support for testing browser-specific behaviours or interactions like JavaScript execution, CSS rendering, etc (the rendering time and JS execution times are browser side and should be the same regardless of the number of users). Gatling still replicates user behaviour very well and can scale to replicate thousands or in the case of the Enterprise version millions of concurrent users.

Selenium is not designed to scale like Gatling but does execute JS and renders CSS from different browsers (this is very resource intensive so part of the reason it doesn’t scale well). The performance metrics also don’t really match up to Gatling’s (especially the Enterprise version).

So they are separate tools for different purposes Gatling is designed for load testing, Selenium is for functional testing.

Cheers,
Pete

1 Like
3

Hi,
Thanks for answering my question in details, I’m happy for knowing Gatling.
To add up, one more question about this:

  • By what metrics does Gatling measure for total response time? You said there were TLS handshake, DNS resolutions… Could you be more specific about this part? Note that I don’t have a strong knowledge about network (which is a completely disaster) so a simple answer / detailed answer is up to your choice, I appreciate both.

Cheers!

4

Hello Trnip,

That was specifically for the Enterprise version of Gatling. The open source version of Gatling reports:

  • Response times (broken down by percentiles)
  • Error rates
  • Active users

The Enterprise version reports the above but also breaks down the response times so you can get more information about each request including:

  • The TLS Handshake time - With increasingly complex bots come increasingly complex TLS handshakes which takes longer time. Knowing the time this takes makes it easier to see whether a load issue is due to the request, network, etc. or the TLS handshake.
  • TCP Connections: used as the transport layer of the HTTP/1.1 and HTTP/2 protocols, this metric represents the time it takes for a server to acknowledge new clients so they can start sending data. this helps show you if accepting new clients on your application is working successfully or causing unwanted issues.
  • DNS: You get the option of examining the DNS resolutions per second, DNS percentiles, DNS duration distribution and percentiles for each hostname your website or application is using. Helps you know is my DNS resolution fast enough? Is my first DNS query for my website too slow? Do I need DNS servers closer to my customers? Are the queries cached properly?

Hope that helps!

1 Like
5

Thanks a lot Pete ! These answers are helpful to me

1 Like
6

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.