This text is a presentation report from a project where we focused on a travel planner. The team´s assignment was to do planning, select test methods, techniques, write test cases, run the tests and write a bug report. It is only focused on the web-based version and only the planner itself.
Planning
Sprint Planning
First thing we did in our planning was to go thru the categories of the project. We split it down to three separate groups: Planning, Test cases and Test runs/ Bug reports. Then we looked at the time we had to work with, from this we decided to do a sort of “sprint planning” and give the three groups 1 week each. The reason for this was to give us a start with an overview that was easy to follow. As we are a group of people it is important that we all can follow what everyone is working with and have the same deadlines.
We also had one person in the group who had internet problems and could not always attend the meetings. To make it easier for him to follow along and be able to feel to be a part of the group we needed to have clear communication. So, everything mentioned at our meetings, timelines, work tasks were written down in a document, available to everyone. All meetings were recorded. We have about 2 meetings a week, and because of the internet problems we all meet thru Teams and then the one group member was called up on phone and could
communicate with the rest of the group even if he couldn´t see the screen.
In our first week we had Planning as a task.
This included the topics:
Area Tour, Functions
We looked thru the product and made a list of all main functions and sub functions( like links, translate etc.). Here we talked back and forth of the complexity of the functions, which was connected, which could be tested separately, what was included and not on the entire page (we have links, header, footer that was not a part of the project to test but still visible on the page).
Oracle
As we did not have any manuals or documentation available, we had to figure out some kind of oracle. During this stage we had a lecture with one of the developers who had been working at Skånetrafiken when they developed this product. There we got the information that Skånetrafiken and Blekingetrafiken had used the same code base. Basically, the only differences were the database. So, we knew that we could not use Skånetrafiken as an oracle.
We decided to look at some bigger cities where we knew their travelling planners was popular and had helpful reviews. Some we selected was London City mapper, Copenhagen travel planner, Västtrafik, and google travel planner. And google maps for the map function.
Models
To be able to split up the tasks in the group we talked about activity diagrams, flow charts etc. We thought it was important to get an overview of the entire project. We started on different diagrams but soon realized that it caused confusion within the group, since many
had never worked with diagrams or flow charts. It felt like it took or time to just figure out and learn that topic. Because of this I made a fast diagram over the sections in the project, so we had a starting point. It looked like this:
And then I split the main categories up and added subcategories, what was needed to be tested.
Complexity Risks
We divided the product into different modules. Looked at what could be tested separately and what was connected. The complexity of the functions. First, we were thinking to only test “in south Sweden” but since the map function wasn´t available there we also decided to test “in Blekinge”. The reason we didn´t include “in Sweden/north Europe” was because all those functions was the same as in the other two. We also looked at the functions separately, to see how complex they were. Ex: if the to and from destination does not work, what is affected. Also, the list and map function, which is not visible at first sight but overly complex functions themselves. We decided that they were needed to be tested separately since their complexity even if the product itself could work without them.
Now afterwards I can think that here was where we did some fault choices, we didn´t do any risk analysis of the functions during this step. It was something we did afterwards instead, if we had done it here, I think it would have been easier for us to prioritize test cases in our individual work later on.
Individual tasks
From the diagrams and module categories above, we had an overview over the project and work tasks. From this we split the functions between us. We took one module each, except module 1, where me and Andrea was going to work together since that was the biggest module and included most test cases. Other modules was: Time/date, travel with, Other preferences, Usability/UI, Translate, Help and new search.
Test Methods
For the test methods we went thru all methods and had a discussion if they would fit in our project or not. In the end the exclusion method made us choose context driven testing as out method.
This for the reasons:
Analytical testing is too complex, we don´t need precises mathematics and analytics for this product.
Standard testing: We did not have any traceability matrix
Quality assurance: No documents of quality criteria was available; we did not have access to the code and can not make a full judgement of what is expected as quality criterions.
Agile testing: Even if we did some kind of sprint planning, we did not work Agile, and was not working with test automation.
Test Techniques
The different modules had different test techniques hat would work better so we decided to make these decisions individually. Some techniques that were used was: Domain testing, Function testing, Risk testing and Flow testing. Domain testing was mostly used in module 3 to check the boundary values between the checkboxes, and module 2, time and date.
Function testing was our main testing technique, we all looked at the functions tested to see that all of them worked as expected.
Risk testing was used to look at severity and complexity of the functions. What was affected in case of problems.
Flow testing was also a big part since the product is designed to connect different choices and give results based upon that.
Test Cases
The group decided to follow the same pattern in title and description for the test cases, this was something we talked about before we started working individually so we all knew what was expected.
Since me and Andrea had the same module, we had meeting thru Teams where we wrote down a list of primary areas to evaluate of the location fields in our module. Here is an overview of some of it:
We made a list of about 25 Test cases. These we divided between each other. I took the once that was including the map, gps and Via functions. One thing that was difficult here was how to think as a tester. How should we think around the test cases, should we make it easy to write the test cases that would all end up OK. Write them as they were experienced on the screen or write it as we normally would expect. As to say, there was quite a bit of visible bugs already from start and it created a good discussion between us and then between the entire group. Our conclusion was that we can not write out test cases from what we see, we need to write them as we expect. How would we otherwise sort out bugs?
I ended up writing 22 test cases. Mostly the map and gps, to evaluate all menu options in the map, to see that is selected the areas correct, to see if it worked to select to and from destinations and so on. Same with gps function, to test the function to see if it worked on both to and from destination. Here it was a bit of a challenge to write the test cases since there was nothing that worked. I tried to activate the gps in all kind of setting, firewalls, hardware, location settings on the computer, double checked the mobile app functions. Thanks to our group conversation earlier where we had decided how to go about these cases I wrote the test cases as I would expect them to work.
Test Runs
All to all the group ended up writing 162 test cases (!).
In the start of the project, we made the decision that we all wanted to learn as much as possible in this project, to make it real and make us do our work as good as possible. Because of this we decided to switch Test cases and Test Runs, so we were going to do another person’s test cases when we did the test runs and bug reports.
I must say that it was surprisingly easy to do this switch. The effective communication in between the group made miscommunication almost nonexistent. At out meeting after all test cases was done, we had a walkthrough in ReQtest and went thru the functions of Test Runs, so we all knew how it worked and we talked thru our questions and helped each other figure out the right process of test evaluation.
Bug Reports and Results
The group ended up with a total of 22 bugs. During this last step we realized that we needed to do the risk analysis. It was a mistake as I mentioned above, it should have been earlier in the process. But we still felt like it was important to do it, both to learn how to think around it and to learn how to set the severity and complexity scale on a function and its bug.
This is our result from that:
Afterwards we had another meeting where we looked thru the list of bugs, compared it to our risk analysis matrix to select some of the most major bugs.
And at last, we also realized that we had not made a regression test suite that could be reused. We didn´t think of the importance of creating one at first. That it is important to have a regression test suite, both to limit the amount of test cases and to easier go back and effectively do the same tests again.
