Approaching accessibility testing through automation

source :

Don’t worry about being successful, but work towards being significant and the success will naturally follow — Oprah Winfrey

Rightly quoted, indeed! A business strives towards success when it generates high revenue, but it achieves excellence when it benefits each individual of the society — irrespective of their physical, phycological or emotional traits.

Accessibility in a software product is one such step — imparting flexibility in an application to cater everyone, including the specially abled people who wish to avail those goods and services. Degree of accessibility defines the ease of using a product by one and all regardless of their respective natural limitations. History reveals that those who’ve believed in creating equal opportunities for all and delivered products at doorstep of abled and differently abled people together are the most successful entrepreneurs.

Web accessibility is not only a moral obligation that an organization carries out in accordance to its social and corporate responsibilities, its also a significant step towards raising the turnovers. Studies have shown that people with disabilities globally have around 1000 billion dollars of disposable income. A survey by National Organization on Disability reveals that specially abled people in the region of North America constitute a market of approximately USD 220 billion. This amazing fact is enough to demonstrate the benefits a business could attain on launching highly accessible products across the globe.

In India, adhering to WCAG 2.0 guidelines and government compliance 508 are 2 major steps which all businesses need to take for creating highly accessible web applications. While developing such a product, teams would need to make sure that all standards under accessibility are met and it becomes possible for specially abled people to explore and avail all services of the web application.

Challenges with Accessibility Testing

Talking about accessibility testing for a web application, it is either considered as a least priority task or is completely ignored in the product development cycle. Few challenges in the whole process of accessibility testing concept and its implementation include the following :

  1. Accessibility experts : A team of specially abled people who are willing and ready to devote time to carry out end to end accessibility testing is difficult to build and maintain. It involves a considerable investment in terms of time and money, alongside a need of following rigorous accessibility guidelines and practicing different set of processes
  2. Time of occurrence : Majority of projects give preliminary prominence to other kinds of testing like unit, functional, integration and others but accessibility testing falls far behind. Majority of the planned time is already taken in other forms of testing and teams end up doing a random, ad-hoc kind of accessibility testing right before releases — and only the critical/blocking things are fixed
  3. Existing approaches : Tools that are in market for relevant testing are mostly time taking and less secure. One such example is Wave tool which takes application’s html content over the web to Wave server for scrutinising the accessibility issues, thus posing threat to integrity, privacy and security of product

Solution — as always, Automation!

While all unit and functional tests are getting automated and executed using the continuous integration tools, what if we introduce automated accessibility tests in our test code and make testing smoother and more comprehensive? A great idea — well, let’s see how could that be possible!

aXe core API — the accessibility engine

aXe is a free, open source accessibility API that allows adding accessibility tests in the existing selenium automation framework. The aXe library is built with all pre-defined accessibility rules and guidelines which are matched against each web page of an application, resulting into a list of accessibility violations on the page, if any. The major benefits of using aXe API include :

  1. Integration with selenium : With selenium-based functional tests running on your application, invoking the accessibility tests using aXe is straight forward. No need to create a separate codebase. aXe API can easily be a part of existing test automation project, thus making it a one-to-one mapping between pages that selenium access and pages that are tested for accessibility
  2. Support for major browsers : aXe works like a great companion to selenium, all browsers that you can automate using selenium are equally supported for accessibility tests with aXe. Hence, popular browsers like Chrome, Safari and FF (also IE) can be tested seamlessly for functional and accessibility issues at the same time
  3. Cost-effective accessibility testing : The software industry relies on a lot of tools that are freely available and highly customizable — one such tool is aXe. Its a solution that adds no license cost to the organization while providing an in-depth testing on accessibility front as and when needed
  4. Accessibility experts not required anymore: Any functional testing engineer who creates and maintains the selenium tests can add the accessibility checks himself, without a need of an accessibility expert.

A sample report from running aXe automated tests would look like this :

5. Increased frequency of accessibility testing : With automation in place, the number of executing accessibility tests become as much as functional tests. With each trigger to functional tests, like at each PR generation on source code, periodic executions using CI tools or local executions while script development, accessibility tests would also be carried out and results be shown to QAs and the devs. This gives more time for everyone to decide upon each bug and prioritize fixing it

How does the aXe API work

To be able to work with aXe api, a framework must have 2 things

a. aXe.min.js — the javascript file having all necessary accessibility rules and methods

b. support of javascript execution within the framework(which every selenium based framework already would have)

The guidelines/rules that aXe library covers can be found at this location

Steps to follow for adding aXe to automation :

  1. Under the project resources/utilities package, add the javascript file
  2. With your selenium page objects, create a wrapper aXe class that defined object using AXE builder and returns the JSON object with all possible violations
  3. In your existing test classes, just pass the driver instance and url to aXe wrapper method and receive the JSON — put the assertions around it

A detailed example of the same can be found here.

Limitations with aXe

There are a few areas that still be can’t covered with aXe, including the following :

  1. Incorrect reading sequence — A page split into 2 columns having content both sides would not be understood by the API — it always tries to read from left to right
  2. Incorrect labels — API can’t judge the validity of a label alongside an input field, semantically
  3. Focus related issues — Those accessibility bugs which arise out of focus on page can’t be analysed by this API

With the given limitations, there is a definite need of manual testing accessibility experts to work hand-in-hand with automation. However, the degree of their effort and time would be saved a lot, on top of which lies the luxury of unveiling accessibility issues quickly with automation.


Automation of accessibility tests would lead to an early and often testing cycle for accessibility in the web product, allowing functional engineers to leverage their selenium automation to the accessibility testing levels. This would result in a continuous accessibility check and help keeping up with all laid down standards.

Want to work with the awesome Tech team at UpGrad? Check out the positions now open!

Want to work with the awesome Tech team at UpGrad? Check out the positions now open!

Approaching accessibility testing through automation was originally published in Technology at UpGrad on Medium, where people are continuing the conversation by highlighting and responding to this story.

Source: UpGrad