Kubernetes and Selenium Grid for highly scalable browser and device farm

The Cloud and the emergence of micro-services pose two challenges to the Selenium Grid based solution for concurrent execution of UI automation tests. Having to keep a tab on the cost of Cloud is also another challenge faced.

First one is, in the world micro-frontends, teams want to run all UI automation tests they have for a project, against every deployment of their Git branches (as part of validating a Pull Requests before they are merged). This problem is multiplied by the number of micro-frontends an organisation has.

Speed will be an important factor in the pre-merge pipeline(Shift Left) as described above. Speed here is the speed of the feedback cycle. What was acceptable last year, is no longer acceptable this year. Hence the speed of the feedback cycle, which is the total time taken, starting from a commit to a Git branch triggering the pipeline, deploying it to a fully stack environment, run UI automation tests against the deployed stack, delete the full stack environment and ends with the availability of the automation results for the said branch.

Hence for the first problem, Kubernetes with Docker and Helm will be the most suitable for pre-merge pipelines.

Regarding cloud cost optimization, Kubernetes gives the flexibility to fine tune CPU/Memory individually at Pod level.

The Kubernetes, Docker, Traefik and Helm based solution will fix all the identified problems above – on demand unique grids per pre-merge pipelines, faster creation/deletion of hubs/nodes, accessible from anywhere, more cost optimized compared to pure EC2.

My talk is about describing the above problem and how I have implemented the solution. I’ll share all the learnings I had when implementing this solution. For example, how to fine tune the CPU and Memory per browser Pod. How to configure CPU/Memory for small scale hubs which will run less than 100 test vs for hubs that will run 300 tests or more in parallel.

Selenium Grid using Kubernetes, Docker, Traefik and Helm will be useful to many companies who are planning to or in the midst of moving to Cloud, so they can have a long term solution and learn from our journey.