Browser Automation: How an efficient Selenium Cloud Platform should look like in 2020
Hi there! During previous years we have been talking a lot about deploying and maintaining an efficient in-house browser automation infrastructure. You can find a full list of these articles on our website. During these years thousands of teams were happy with Selenium infrastructure tools we created as everything we deliver can be deployed in 5 minutes and has zero effort maintenance. However many companies still prefer using Selenium cloud solutions like SauceLabs, Browserstack, TestingBot, CrossBrowserTesting and so on delivering browser automation infrastructure as a service. So if in your company you are already using some cloud solution for testing in browsers or just planning to use it — this article is for you.
What’s Inside Traditional Cloud Platform
Although nowadays there are several popular cloud platforms with browsers — all of them provide very similar set of features:
- Live Testing. That means you can use any available browser version like you are using your own browser to do some manual checks: opening pages, clicking on the buttons, typing in input fields and so on. Selected browser version is running remotely on cloud platform computing resources and its desktop is shown in your browser using remote desktop access.
- Automated Testing. This is the same feature as live testing but all commands to browser are being sent by a program via a remote API. The most popular browser automation standard nowadays is called Selenium WebDriver, so all cloud platforms only support commands corresponding to this standard.
- Taking Screenshots. That means you can insert a list of URLs to the form, select a desired browser version and cloud platform will automatically take and store the screenshots for you.
- Screenshot-based Regression Testing. That means you can easily test that your changes did not break your web-site. This is done by first taking a screenshot of correct web-site layout and then comparing a screenshot of changed web-site with correct one pixel per pixel. Such approach allows to easily detect even small shifts in rendering invisible to human’s eye.
Pricing model also remains pretty much the same from one provider to another — on every billing plan you have a fixed number of browsers that can be launched in parallel. In some cases you are also limited by the total time you can use available browsers. Payments are usually done on monthly or yearly basis and do not depend on your real consumption.
So what’s wrong with these platforms? According to our users having Selenium public cloud platforms experience there are several really annoying points:
- Limited parallel browsers. This limitation assumes that you always need the same number of browsers and running them every day. In real life browser usage varies a lot. You usually need browsers during the working days but may not need them at the night and during the week-end. While preparing the release you would usually prefer having as many browsers as possible to finish testing faster but 90% of time so many browsers are useless. So teams usually want to have an unlimited number of browsers available.
- Paying for the time when browsers are not running. A lot of teams confirmed that they are running tests requiring browsers e.g. one hour once per week. Except this hour browsers are not needed. Monthly and yearly payments make no sense with such usage scenario. Such teams would prefer pay as you go billing depending on real browser consumption.
- Very slow and unstable network connectivity. This includes two types of issues. The first one is big distance between remote browser location and user location. For example your Selenium cloud provider can be running in a datacenter in the United States whereas your test is being launched in Europe. Because of that every command to browser will be executed slower and overall test execution time increases dramatically.
The second issue is network connectivity between browsers and your corporate network. Very often you have to open a web site that is accessible in your corporate network but is not accessible from the Internet (e.g. access is blocked by firewall). So far as cloud provider browsers are running outside of your corporate network you have to explicitly configure network access from browsers. Cloud providers usually solve this problem by providing so-called “secure tunnel” feature. In order to allow a remote browser opening your corporate network sites you have to download and run a special program on a machine inside your network. This program is automatically connecting to cloud provider and then starts forwarding all network traffic between the browser and desired web site. Usually this is implemented either with SSH port forwarding or with forwarding TCP traffic through web sockets. Nevertheless such tunnels are known to be slow, unstable and require additional preparatory steps in your tests.
4. Security concerns. Public cloud providers declare that their browsers are secure and tunnels are completely isolated. However there is no way to check this. Nobody can check how good is isolation between browsers belonging to different clients. All commands to browsers are going through the same API — something like this:
What if there is a security vulnerability in this API and one client can read commands corresponding to another client?! Also all data like screenshots, logs and videos showing what’s being tested is stored on cloud provider side and you can’t be sure that this data is really removed when you request it.
Introducing Moon Cloud
Having all these complaints we reconsidered how a modern browser automation cloud should look like and introduce a new solution called Moon Cloud:
Out of the box you get a dedicated Selenium installation running in one of cloud platforms of your choice: Amazon Web Services, Google Cloud or Microsoft Azure. To access browsers you have to use an address like
https://<your-company>.cloud.aerokube.com/. Every cluster in Moon Cloud is maintained by our engineers and supports both automated and live testing similarly to what you could see in another platforms. So migrating existing tests from another cloud platform is as simple as changing Selenium URL in tests.
To understand how Moon Cloud differs from other platforms and why it is more flexible let’s check what’s under the hood. Moon Cloud is certainly using Moon for running browsers. Moon on its turn is running inside Kubernetes cluster. In public cloud platforms Kubernetes is usually started on top of one or several virtual machines. These virtual machines can be automatically added or removed depending on the load and thus dramatically reduce computing resources cost. This feature is called auto-scaling. Because of auto-scaling you can easily get as many computing resources as currently needed. In terms of browser automation that means you can run an unlimited number of browsers in parallel. How many browsers to run is now your decision and can vary a lot depending on your development phase. Another important change is that with such architecture you now pay as you go. Cloud platforms now use per-minute billing for the majority of resources they provide, so actual browser automation infrastructure cost will also depend on your usage pattern.
Having a cost-efficient Selenium cluster let’s now address network connectivity, reliability and security concerns. First of all every cluster in Moon Cloud is dedicated. That is to say it is always running on independent virtual machines and is using an independent load balancing infrastructure and domain name. All your commands to browsers go through a TLS encrypted connection. All data generated by running browsers such as logs and recorded video files can be optionally saved to completely independent S3 bucket. If you wish to fully control your data this S3 bucket can be created on your side. Moon Cloud contrarily to other cloud platforms is always running at least in two datacenters and thus continues to work even if the entire datacenter goes down because of power or network outage. To deliver maximum network performance between browsers and your tested environments Moon Cloud is always running in cloud region of your choice (e.g. US East, US West, Europe, Africa, Australia and so on). Because of that browsers are lightning fast.
Instead of ugly tunnels Moon Cloud relies on networking features provided by cloud platforms, e.g. Virtual Private Cloud. This includes three main cases:
- Your infrastructure also runs in cloud platform. In that case to provide network access virtual network of Moon Cloud is given access to virtual network where your application is running (so-called VPC peering).
- Your infrastructure runs on your own servers. In that case corporate network can be connected to cloud platform. For example possible options for AWS cloud can be found in article.
- You need to test something running on your workstation. This is a particular case of 2) and can be easily solved by configuring permanent VPN connection from workstation to cloud platform. That’s it! No more need to start executables from untrusted publishers.
As you can see Moon Cloud already solves all the complaints from traditional cloud provider users described above. But we have one more thing — browser automation nowadays does not equal to Selenium. Companies like Google and Microsoft are already working on next-generation browser automation technologies: Puppeteer and Playwright. These frameworks deliver faster browser automation and have powerful features that simply do not exist in Selenium. Take a look at our articles describing them in detail:
- Playwright: Launching Cross-Browser Automation to the Stars
- Moon: the Swiss Army Knife for Browser Automation
Moon already has first-class support for parallel execution of Playwright and Puppeteer tests. And so does Moon Cloud. Thus you get a true Swiss army knife of browser automation compatible with the most efficient frameworks existing nowadays.
Moon Cloud Pricing
Two last questions we certainly need to cover is Moon Cloud pricing and what to do if later you decide to run browser automation on your own hardware. The pricing model is pretty straightforward. Moon Cloud is always running in public cloud having its own billing. Our pricing model is adding a fixed interest rate of 25% on top of regular cloud billing. For example if cloud provider bills us $100 monthly then your final price will be $125. Because of such flexible billing model there is no yearly payment option — it simply makes no sense. Finally if for some reason later you decide to move to your own hardware — then you can simply deploy your own Moon cluster in minutes. The only change in tests in that case will be an URL to obtain browsers. So whatever you decide — migration will be effortless.
If you are interested in trying Moon Cloud — we would be glad to provide you a free trial as well as a free dedicated support channel. Simply contact us by email firstname.lastname@example.org and stop wasting your money!