Based on the State of Agile report in 2021, Scrum was found to be the most commonly-followed methodology for agile teams. Scrum is an iterative-based agile methodology introduced by Ken Schwaber and Jeff Sutherland, who developed The Scrum Guide to create a definition around the idea of Scrum and its processes.
In this article, we will take a closer look at the challenges and the solution for implementing a robust Software Testing practice in Scrum, ensuring that your team is producing a quality product every time they deploy.
1. There is no Software Tester Role in a Scrum Team
According to The Scrum Guide, the scrum team is a cross-functional team, and in this structure, there is no specifically defined Software Tester role. All the team members, other than the Scrum Master and Product Owners, are considered as Developers. In resourcing a specific Scrum team, some organisations would opt to include a dedicated software tester in the Developers team, while some will not and may expect the developers to be the ones to take on the software testing role. If software developers are not equipped with software testing skills and specific experience in this field, then this could quickly become an issue, particularly for teams building complex products.
While ideally, we would like to have a software tester be included in the team, if we are utilising Software Developers as testers, then there are several options we have to reduce the impact of inexperienced software testers:
a. Define product back log items really specifically, including adopting the User Story format, using the “As a…” “I want to…” “So That…” format.
This format defines the software requirement purely from the user perspective. By using this format, we are also forcing the Product Owner to state on why the required feature is needed (or tracing the feature to the user requirement). This format will make it easier for the Scrum team member to design test scenarios and ensure that they are testing for a real-life use case rather than for a hypothetical, ensuring that any functionality is as per the user’s expectations.
User Story Example:
- As a car owner, I want to complete my user registration, so that I can post a free advertisement for selling my car.
- As a homeowner, I want to simulate my mortgage payment, so that I can determine what my mortgage repayments will be.
b. Define acceptance criteria in the product back log by using the “Given-When-Then” format. Known as the Gherkin format, where the acceptance criteria has been defined using “Give -When-Then”, allows developers to understand the testing requirements and use cases to ensure that developers can test features based on a real-world user scenario.
Acceptance Criteria Example:
- Given the visitor has accessed their user registration through the main page, when the visitor submits their user id, password, name and last name, then the visitor can login to the system using the newly created user id.
- Given the loan term is 60 months, home value is $50k USD, down payment is $10k USD and interest rate is at 12% annually, when the user simulates the monthly payment, then the system will provide a calculation of the estimated mortgage repayment. (example updated, so it made sense with the one above)
c. Train the Developers team in software testing. There are free training courses available online, such as the BBST training material provided by Cem Kaner. The basic content, designed for beginners including the fundamentals of testing, bug reporting and some basic test design, can be covered in just a few days of training with some supervision.
2. Pressure to deliver in short time frames
The length of a Sprint in a Scrum is usually two weeks, although The Scrum Guide states that the maximum Sprint duration can be up to a month. This puts pressure on the development team to get things done quickly to satisfy all of the ‘Definition of Done’ requirements and quality measures for each individual Product Backlog committed to in that Sprint.
Specifically related to testing, there are some issues that usually come with this frenetic page:
a. Regression Testing takes time
As the developed product grows more in size, to do complete regression testing (and bug fix for any bugs found in that process) may take more effort and could cause delay for a sprint with a short duration.
To mitigate this risk, we can usually rely on more Test Automation for the repetitive testing that needed to be done throughout the product. However, not all organisation are equipped with the skill set and resourcing to do this properly.
Test automation is usually applied at various parts of a system, such as:
- Unit level
- Component level
- Integration level
- Front end
Test automation often focuses on testing the functionality and performance of a single part of the system. Penetration testing and installation testing can also be done through Test Automation, although again, appropriate skills and resourcing is key for this to be successful.
Not all kinds of regression testing are best performed through test automation, and there are some tests that may be better to be done manually. Test Automation is not a catch-all for testing, and needs to be applied carefully in your testing phase.
b. Testing (and bug fixing) the Product Backlog committed for the Sprint takes time
To properly develop, test and bug fix a feature or requirement may take more than two weeks; this is a challenge if the Sprint duration is also only two weeks!
To manage this, you need to ensure the size of the Sprint Backlog has been broken down to small manageable pieces so that proper testing and bug fix can be done prior to the Sprint Review. By working off smaller testing parts, you reduce the amount of time taken per test and per bug, allowing you to resolve issues more quickly than if you were testing a whole feature each time.
3. Tracking Bugs in Scrum
Most agile project management tools can be configured to act as a Scrum board, Kanban board, or bug tracking board; however, usually there is no option for the Scrum board to be integrated with bug tracking, even though there are often bugs found during the Sprint phase.
To mitigate this, it is best if the bug related to the Sprint Backlog item is tracked as a task from within the Backlog item, allowing you to keep all notes and requirements for a Backlog item in a single location. If the bug is not related to a current Sprint Backlog Item, we can create a new Backlog Item for this task in the Scrum Board.
4. No Test Plan required in a Scrum Project
While it seems there is no formal requirement to come up with Test Plan based on The Scrum Guide, it is important to find its equivalent within Scrum Methodology.
In Scrum Methodology, Definition of Done is defined for a group of product backlog items (or an increment) as the quality plan for the product increment.
To achieve ‘Definition of Done’ in Scrum, not just for coding activities, the Scrum Team must perform testing and bug fixing activities to ensure the items detailed in the ‘Definition of Done’ can be finalised. As therenot much guideline from The Scrum Guide on the type testing that we need to do across Agile projects, we can instead add and adopt the Agile Testing Quadrant as a guideline to plan for testing in Scrum projects. The Agile Testing Quadrant was created by Lisa Crispin and Janet Gregory based on Brian Marick’s post on the Agile Testing Matrix.

According to Scrum Methodology, by default only some of Q2 and Q3 tests are covered during the Sprint Backlog items ‘Definition of Done’, and during the Sprint Review and product presentation. Q1 tests can be missed if not included in the ‘Definition of Done’, and Q4 tests are usually just an afterthought during the development process. By considering the product release plan, we can use the Agile Testing Quadrant to remind us of the type of testing appropriate across the project so that these tests can be properly planned and incorporated into the Scrum process.
Q1 tests are usually automated, while Q2 tests can be automated. Q3 tests are executed manually by experienced tester and product owner. Q4 tests are usually automated and completed using special tools and reporting dashboards. When tests can be automated, we can combine them with the Continuous Integration (CI)/Continuous Deployment (CD) solution.
5. Utilizing Dedicated Testers in Scrum
Fully utilizing dedicated testers in scrum team sometimes can be a challenge especially in the early development where the testing tasks could still be limited and not much regression testing needed.
To mitigate this issue, we can utilize the testers as the following:
1) Assist product owner to improve the backlog items definition and acceptance criteria.
2) Automate test (API test, mock up test, Test through UI)
3) Perform non functional testing
4) Assisting scrum master in scrum and agile practices
So that software testers can be more effective in the scrum team they can be equipped with skills in Scrum Process, Agile Practices, Test Automation, DevOps and Software Requirement.
We can also add more testers to the scrum team at the later sprints as there are more testing tasks especially for regression testing that are needed to be done within the sprint.
Mitrais Software Testing Service
At Mitrais, we have adopted Scrum as the default software development methodology, and our Software Tester and Software Developer roles within a Scrum Team are trained in performing a wide range of software testing activities, despite their exclusion from the standard Scrum Methodology. We work with organisations across Asia Pacific to expand and augment their local teams with specially trained, highly competent, and dedicated Software Developers who are conversant in the latest methodologies and technologies, to ensure you deliver a fantastic product every time. If you need any assistance in performing testing in agile projects, you can contact us through our website.