Crossbrowser Testing

What is Cross browser testing?

As we are moving into an era where everything is connected to web. So that means lot of Web and Mobile applications are being developed everyday. With this growing rate of applications in market, some of the applications become non responsive with different browsers and device form factor.

Testing is very important part of application life cycle. Most of the developers just perform testing on one or two browsers and ignore the fact there are many different web browsers out there.So performing testing on different browsers is called Cross Browser Testing.

There are many Different Desktop and Mobile browsers with many different versions available to download, so it becomes very important to select the right ones and perform testing.

As we move along with this article, you will get know how to choose the right browser combination for your application.

Why it is Important?

Cross browser testing has become an important part of  software testing (web and mobile). Applications developed should be tested on different browsers and different resolutions so that there is a seamless user experience.

Many users turn down or share negative feedback of application as whole or some part of application is not responsive on different browsers, for example : most of the applications do not load properly on Internet Explorer. So to avoid this Cross browser testing is a must.

How to Start with Crossbrowser testing?

To begin with Crossbrowser testing we need to keep few things in mind.

• Analysing the correct requirement
• Testing on right browser.
• Methodology
• Analysing the results

Analysing the correct requirement

Now to begin with, we first need to analyse our requirement for crossbrowser testing.
We have to keep few things in mind for doing so:

• Nature of application (small, medium or large)
• Target user coverage
• Limitations of application
• Impact on budget
• Avoid under and over testing
• Type of testing required

Nature of application

We need to know the nature of our application and how big is it. Applications can be classified as small, medium and large.

Small Scale : Mostly with less than 10 pages and minimal dynamic content

Medium Scale : Mostly with 10 to 50 pages with dynamic content

Large Scale : Mostly enterprise application with 100+ pages.

Target user coverage

We need to estimate the area of coverage i.e percentage of end users you want to engage.

Limitation of application

Some applications have limitation such as compatible with certain browsers. So in this case compatible browsers should be tested and a compatibility alert should be given to end user if any other browser is detected.

Impact on budget

We need to make sure our budget should not overflow. As testing Cross browser may involve certain infrastructure and testing effort. 

Avoid under and over testing

Application should not be under tested as it will result in poor user experience, It should not be over tested as it will result in more money investment.

Type of testing required
There can be different type of testing you may perform. example function testing, visual testing etc.

Testing on right browser

 The best way to select right browsers for your application is by there market share in current year and previous years. We will use previous year data because there is a small chance that some of the users might not have updated the browser.




To know the market share of different browsers we can refer to below table (for year 2017 and 2016). If you are looking this post in future then please refer this link.

Year 2017

2017	Chrome	IE/Edge	Firefox	Safari	Opera
October	76.1 %	4.1 %	12.1 %	3.3 %	1.2 %
September	76.5 %	4.2 %	12.8 %	3.2 %	1.2 %
August	76.9 %	4.3 %	13.1 %	3.0 %	1.2 %
July	76.7 %	4.2 %	13.3 %	3.0 %	1.2 %
June	76.3 %	4.6 %	13.3 %	3.3 %	1.2 %
May	75.8 %	4.6 %	13.6 %	3.4 %	1.1 %
April	75.7 %	4.6 %	13.6 %	3.7 %	1.1 %
March	75.1 %	4.8 %	14.1 %	3.6 %	1.0 %
February	74.1 %	4.8 %	15.0 %	3.6 %	1.0 %
January	73.7 %	4.9 %	15.4 %	3.6 %	1.0 %

Year 2016

2016	Chrome	IE/Edge	Firefox	Safari	Opera
December	73.7 %	4.8 %	15.5 %	3.5 %	1.1 %
November	73.8 %	5.2 %	15.3 %	3.5 %	1.1 %
October	73.0 %	5.2 %	15.7 %	3.6 %	1.1 %
September	72.5 %	5.3 %	16.3 %	3.5 %	1.0 %
August	72.4 %	5.2 %	16.8 %	3.2 %	1.1 %
July	71.9 %	5.2 %	17.1 %	3.2 %	1.1 %
June	71.7 %	5.6 %	17.0 %	3.3 %	1.1 %
May	71.4 %	5.7 %	16.9 %	3.6 %	1.2 %
April	70.4 %	5.8 %	17.5 %	3.7 %	1.3 %
March	69.9 %	6.1 %	17.8 %	3.6 %	1.3 %
February	69.0 %	6.2 %	18.6 %	3.7 %	1.3 %
January	68.4 %	6.2 %	18.8 %	3.7 %	1.4 %

If we see the above tables, data shows the percentage of users per browser.
Now not every application needs ~100% coverage. It depends on the nature of application and area of use. Testing should give the optimal coverage with less cost, so to achieve that we will follow below mentioned simple selections

For small scale application we will try to achieve ~ 90% coverage

This can be achieved by selecting only 2 browsers i.e Chrome 72.1% and Firefox 12.1%
By testing on these two browsers we can give our application ~90% coverage

For medium scale applications we will try to achieve ~ 95% coverage

This can be achieved by selecting 3 browsers from the above tables i.e Chroma, Firefox and IE/Edge.
Total coverage will be 92.3%.

For large scale applications we need to cover as much as possible

So for large scale applications we will use minimum of 4 browsers. Chrome, Firefox, IE/Edge and Safari.


This will not give us optimal coverage as different browsers have different version so to achieve  that we will further scale it version of different browsers. 

Now question is how do we select which version is best for testing?
Answer to this is minimum 2 version of each browser and maximum 5 versions. This should not include any beta release. We will select 2 to 5 stable versions with maximum end users and addition latest beta version.

How do we know which version is most used?

For Google Chrome : refer this
For Mozilla  Firefox : refer this
For IE/Edge : refer this
For Safari : refer this

Doing this will ensure us maximum coverage and happy end users.

Note : More coverage is equal to good end user experience.No fixed rules. If you are testing small scale application and you have required infrastructure to achieve more coverage go ahead and do it. These rules are with minimum infrastructure requirement as to keep budget friendly.

Methodology

Here we will get know how to start Cross browser Testing. The main idea behind is that we need to have different browsers installed to perform a test.
The best practice is to setup a selenium grid with different browsers installed on different nodes or there are many cloud bases companies which provide you with this setup.
Question here many tech junkies ask is, Can we use single machine to perform Cross-browser testing.
The answer to this is yes we can, but the only drawback is that if you have lot of tests to be executed in parallel. It will slow down the process and you might not end up doing Cross-browser testing.

There are two different ways to do Cross-browser testing.

1. Own Setup.
2. Lend a Setup.

Using Own Setup

• Setup a selenium grid.
• Create nodes with different browsers and different browser versions.
• Write you test suite.
• Execute on local grid setup
• Compare results.
• Run regression suite after every release.

Using a third party Setup

Before proceeding further we should know which companies provide infrastructure. Here are few famous ones. 

And there are many others. So to start with, buy a plan of your choice from one of these guys (I would recommend BrowserStack as they have best offerings) and follow the steps. 

• Use Steps mention in there docs to setup on cloud Grid
• Configure with different browsers as per your requirement.
• Write you test suite.
• Execute on cloud.
• Compare results.
• Run regression suite after every release.

We use this method as most suitable as it saves us the infrastructure cost and once our product is ready and deployed we dont have unused machines.

There are few limitations in using on cloud setup.

1. They do not support Robot Framework

2. You get limited parallel execution slots as per your plan.

That's all folks. Hope you liked the article. Please share your feedback so that i can bring you more stuff on testing.

Next coming up is Cross-browser visual Regression testing(Look and feel testing).