This post is from my assignment within the course with the same name as the post title. We got five requirements to follow when writing it. They were:
Describe a testing tool, it´s purpose and limits
Describe the process of Continuous Integration and discuss pros and cons.
Explain the concept of test automation and how it is connected to BDD
Describe the purpose of version control and briefly explain how it works with git
Describe the process of selecting a test tool
Software testing tool – Purpose and Limits
Axe-Core
My choice of testing tool to explain here is within the topic of web accessibility and the testing library Axe-core. Or more specific Axe DevTools, but we will get back to that. First a brief explanation of axe-core. Axe-core is an open-source JavaScript library, it tests HTML-based UI´s, it is based on a couple of different rules, WCAG 2.0-1 and level A & AA. But the difference from most other accessibility testing tools it also includes other rules, like the most common accessibility practices in different HTML-tags.
Deque which is the company who has built Axe has a couple of different tools built based on Axe-core library, that we can use to evaluate accessibility.
We have axe DevTools Free and Pro, the free version is a browser extension that we can use with Chrome, Edge and Firefox.
Here I ran Axe browser extension on Test Automation University’s website:
As we can see there this page has not done their testing on accessibility. (some embarrassing irony here as this site is supposed to feature professional testers and education). The extension covers a lot of issues and is a good and fast tool but if we look at this newly released report, we can see that automation can do a lot of work for us here:
Read the full report here: Link to report
So, with that said we can use DevTools Pro and write automation tests, unit tests. It comes with the ability for several different integrations like Selenium WebdriverJS. Axe runs a series of tests to check accessibility of both content and functionality on a website, although it only supports open shadow DOM. We can adjust level of impact, contrast, and other standard configurations to preferred test levels. Example of basic unit test with assertEquals test:
And the purpose of why we should test for accessibility in browsers and code? Why this isn´t a job that should be put on the developers of the assistive technologies? Simple answer, we can automate testing in browsers, but not assistive technologies.
We can use these tools to find: Lacking keyboard support, Missing labels, invalid ARIA attributes, color contrasts and more.
The limits of these testing tools are first the lack of knowledge among people in the development business. The decision makers don´t put it as a priority, it is not included at start, which makes it more difficult than it needs to be. The developers think the lack of an “alt=.” or similar in not important. They don´t know what to include or why. Tester’s don´t know what to look for or how to report the issues. But for the tool itself the limits are that we can´t test everything by automation, and as mentioned above some DOM related settings cause limits. It is also a big different depending on what we test, a website and a pdf-file is very different. And one tool can´t do it all.
Continuous Integration, CI – Pros and Cons
Is a development practice where developers integrate changes into the mainline as often as possible. These changes include all types, a function, new code, deleted code. We do this to prevent bigger problems to develop, and in worst case stop the code from running. In Continuous Integration we have the process of plan, code, build, test. It also improves the test reliability due to smaller easier traceable changes. The practice makes the process run smooth and have a structure that is easy for the entire team to follow.
The cons in this practice is that all included in the team has to follow it and be alert of incoming problems. If it is not synchronized with Continuous Delivery the process will come to a halt here and can cause the deployment to be delayed. It also depending upon automated testing which makes it important to manage the planning for the needed manual testing. So, my conclusion is that it is beneficial to manage bugs and keep a good speed in the pipeline. I can also see that it can inspire the team and their confidence when we always can follow the improvements. But it must have a reliable planning behind it.
Test Automation – What is it and its connection with BDD
What is Test Automation
Test automation is when we run the test automatically, managing the test data and do a quality assurance measure. An automated software testing tool is being used and it can enter test data into the system during tests, to compare expected and actual results and generate a test report. There are some sayings that automation testing is done by little to no human interaction. Here I would like to put it that, that is a matter of perspective, humans still write the testing code, do the updates, and still need to do manual control testing. Automated tests help us to not need to sit and run repetitive tests manually, once we have the test cases written we mostly don´t have to interfere.
It helps increase test coverage, speed and can be set to run any time of the day. The process of automated tests is to select a test tool, define what is it we want to automate, plan, and develop the code for the tests. When that is done the tests are done for execution and mostly small maintenance is needed.
Behavior Driven Development, BDD
BDD is a method to write test cases in natural reading language, plain English. It is an effective way to involve all part of the team and not only people who have knowledge in reading code. Test scenarios are often written in Gherkin languages where we use the concept of Give, And, When, Then keywords. Different from TDD where we write failing test code, in BDD we write failing features. Since the scenarios are defined as specific outcomes in a user story we can work with small iterations, which gives a neat and readable documentation. The scenarios are reusable, and we can write different testing combinations to be run. The structure of BDD and scenario features makes it easy to translate it into automation test, we know the requirements of the test cases are met and now we can start the TDD development and write unit tests to apply to our continuous integration pipeline.
Version Control – Purpose and Git process
What is Version Control and its purpose
Version control is a part of software configuration management. It is used to track and control changes to source code, documentations, and configuration files. It is one of the levels in Agile testing, we have: Agile Development, Continuous Integration, Continuous Delivery, Continuous Deployment, DevOps. And included in the Continuous Integration is: Plan, Code, Build and Test.
It makes it easier to control several aspects of the development. For collaborations it makes it easier for the entire team to have access to the source code, keep track of the project and its process, it improves the visibility. Everything is stored and we can see the source code history, which means we can check different versions, if specific commits has been made or needs to be done, which can help accelerate product delivery. We can revert errors, like if we need to troubleshoot an issue, we can use the version control timeline to access and compare working files. And of course, most of all it is an important backup, computer crash, server goes down, anything can happen but if the version control is managed as it should the loss will not be as big.
Version Control process in Git
First a Repository is created. Repository is the main storage location that holds the entire source code and all versions of the software.
To be able to keep track of each developers work and to be able to add code without affecting the main codebase, we use different branches. This can be structured in diverse ways but most common is for each developer to have their own branch where they work until the task is done. It can also be structured that each task has its own branch. So, before a developer starts working, they create and run the command git checkout into their own branch.
Source code is the language-specific code base, where we can track, move, change code. And to get access to existing source code the developer must either clone or fork the code. To clone is simply that the developer copies the entire repository to their local computer. This is done when the purpose is to continue working and contribute on the same project. If the purpose instead is to use the source code as a starting point but make it into a new project, then the fork command is used instead to create a new repository.
The most common git commands to add and receive code from git is:
Git Commit: When we have changes on our local computer, we first need to save these as a version of the repository. This is done by git commit. When we have that saved, it is ready to be added to git, our online storage repository(where our source code is).
Git Pull: To be able to add our commit changes up to git we need to be up to date with the version that is the current online one. To do this we use the git pull command. Since this needs to be done before we can add our changes it is good practice to always try to pull code from git often and before we do any work on our own machine. This keeps risks for merge conflicts lower and saves time. We can also use the command git fetch, this will collect the code from git, but it will not add the code into our local copy.
Git Push: We have created our commit; we have pulled down the current source code from git. Now we are all up to date and can push up our local changes to git.
Git Merge: As mentioned above, we checkout and work in our separate branches. This means that once we have pushed our changes to git, they are added to our online version of our own branch. Now we need to add this to the main branch, so the code is added into our project, for all of the team to easily collect it and to get the changes out to deployment. For this we use the command git merge, if our code is up to date and the rest of the team has followed the version control process properly everything should go smooth. But if there are problem like to files has code that clash, we will get a merge conflict. This needs to be managed before we can add our code into the main branch. For this and for the person responsible for checking the process of merging the code together we create a pull request, this is where a code review is done. The code is checked for quality and security to not cause issues in the main branch.
Strategy to select the right software testing tool
Before we can select the right software testing tool, we need to know our project. Our decision needs to be made upon what is right for the project, our requirements, not what is the best tool.
Look into the key criterions for the project:
What system do we run on, what tools support our technologies etc.
Investigate the skills of the team, do we have enough knowledge for automated tests, is the learning curve realistic for our timeline.
What is within the budget, can we reuse it in the future.
Interaction with other testing tools since we might need to integrate with test management tools, CI, DevOps.
Are the logs and test reports good enough, easy to read test execution analyses.
Is our project and test cases new and not been tested manually yet, or are they frequently changing, then they should probably be seen as non-suitable for automation.
When should we consider test automation necessary: If we have tests that are difficult and become time-consuming to test manually. Also test cases that are critical with high security risk. Repetitive test cases both with smaller and larger amount of data can be beneficial to automate.
Do an analysis of pros and cons of the various tools.