Ensuring the quality and reliability of software applications, software testing serves a pivotal
role with three primary objectives:

• To ensure the quality of the product that we develop.
• To detect defects, bugs, and other issues in the product before it is released to end users.
• To reduce the cost of bug fixings in the product development lifecycle.

In the software testing field, there are many testing approaches and methodologies that are used by software testers to accomplish these objectives, one of them being exploratory testing.

What is Exploratory Testing?

Among the various testing methodologies, exploratory testing stands out as an approach that emphasizes human intuition, creativity, and adaptability. Exploratory testing is an approach to software testing where testers simultaneously design and execute test cases, while learning from the system’s behavior, with minimal reliance on pre-defined test scripts or documentation. It involves active exploration, investigation, and experimentation with the software application to uncover defects, vulnerabilities, and user experience issues that might go unnoticed by traditional scripted testing.

Optimal Timing for Exploratory Testing

Exploratory testing can be conducted at any stage in the development lifecycle. However, let’s discuss this by using an example of a typical project.

Let’s say we are developing a feature to purchase an item from an online shop application.
To start developing this feature, someone responsible for the product, such as the Product Owner or the Business Analyst, will gather and document the product requirements. Then, once the product requirement document is ready, it will be shared with the development team, including the software testers.

Based on this case, here are some examples of when to use exploratory testing.

The Requirement Phase

While reviewing requirements, testers can anticipate potential issues or unclear aspects that may not be explicitly outlined. This process, known as exploratory testing, occurs in the early stages of development. As a tester, you can contribute by reviewing the requirements and using exploratory testing techniques to identify potential ambiguities, inconsistencies, or gaps. By exploring the requirements from a tester’s perspective, you can offer valuable feedback to ensure that the requirements are clear, testable, and aligned with user expectations.

The Development Phase

In the Development Phase, once the requirement document is finalized, developers commence feature development while testers concurrently draft test cases for future testing. During test case creation, testers may identify ambiguities, inconsistencies, or gaps in the feature. Employing exploratory testing, they scrutinize various scenarios to uncover flaws.

For example, when a user wants to proceed with a payment, a user receives a ‘One Time Code’ via SMS to complete the transaction. The application includes a ‘re-send’ button for requesting a new code. However, the absence of limitations in the requirements raises concerns. If a user repeatedly requests the ‘One Time Code,’ it could lead to SMS spamming. The requirement specifies that clicking the ‘re-send’ button generates a new code, automatically expiring the previous one. Nonetheless, there’s no mention of restricting multiple requests, prompting the need for clarification.

The Testing Phase

When the product is ready for testing, you will execute the test cases that you created during the development phase. By doing this testing, we can assume that you already have some expectations on what would be the outcome of each test case that you are executing. If the result does not meet the expectation, then it will be considered a bug. However, there is still a chance that you might have missed some possible test cases during the test case creation if you only rely on the results of the scripted testing. Hence, in this case, you can utilize exploratory testing to do some experiments on some possible test cases that have not been covered in the scripted testing.

For example, in the purchasing feature, you have already tested using these test
cases:

• Fill in the required customer information fields (e.g., shipping address, billing address) and validate that the input fields accept valid data and handle errors appropriately.
• Proceed to the final confirmation step before placing the order, verifying that the order summary, including the selected product, quantity, and total price, is accurate.
• Simulate different scenarios during the purchase process, such as attempting to purchase an item with insufficient stock, using invalid payment details, or canceling the purchase midway.

Now, let’s assume you are already satisfied with the test results, and the test cases are working as expected. However, when you conduct exploratory testing using your intuition and creativity, you might discover some other possible test cases. First, consider who will use this feature; then, imagine what the user will do with this feature. Also, think about what will happen to the app in case of interruptions.

For instance, a user is currently utilizing the feature on an app to purchase a product. When they are about to confirm the payment, an incoming call interrupts the process, or the device’s battery runs out, causing the device to turn off. Therefore, in this phase, it would be beneficial to conduct these tests as part of exploratory testing after completing scripted testing. This approach can help increase your test coverage.

The Post-Release and Maintenance Phase

Finally, during the Post-Release and Maintenance phase. Even after the feature is released, exploratory testing remains valuable. You can continue exploring the software, focusing on edge cases, unexpected user interactions, or performance scenarios. By engaging in exploratory testing during the maintenance phase, testers can uncover any regression issues, evaluate the impact of updates or patches, and maintain the overall quality of the software over its lifecycle.

Advantages of Exploratory Testing

Increasing the Test Coverage

This means we can perform exploratory testing after the scripted testing is completed. So, by performing the exploratory testing, we hope that it can cover parts that the scripted testing might miss. Taking the example from the purchasing feature above, we might notice that there are some interruptions that were not accounted for in the scripted tests, such as an incoming call, internet cut-off, or low battery. It would be difficult to write scripts to emulate these conditions so exploratory testing can complement the scripted testing by helping to uncover issues that were not covered by the scripted tests, and by providing additional insights into the behavior of the application in real-world scenarios.

Gaining More Information About the Product

By exploring the product thoroughly, the software tester can gain more knowledge which is useful for improving the test cases for the scripted testing. This process of exploration can reveal important information that may not have been discovered through scripted testing. By gaining a deeper understanding of the product, testers can provide valuable feedback to both the Product Owner and the development team, improving the overall quality of the software.

Finding Usability Issues in the Application Under Test

By exploring, we position ourselves as the end user, and by doing this, we might uncover usability issues that could happen to the end users in certain circumstances. We can identify areas where the user experience can be improved, such as navigation, workflows, and overall ease of use. We can also identify areas where the application may be confusing or difficult to use and provide feedback to the development team on how to improve the user experience. Additionally, exploratory testing can help identify areas where the application may be inconsistent or not aligned with the user’s expectations and provide recommendations for improvements.

Disadvantages of Exploratory Testing

Varied Results Based on Each Tester

The results of exploratory testing can vary depending on the software tester’s skills, experience, and intuition. For example, it could be that a software tester with years of experience finds more bugs or defects than a software tester with months of experience when testing the same feature.

Lack of Documentation

As exploratory testing relies on ad-hoc test design and execution, it may lack detailed documentation, making it challenging to reproduce exact steps for future reference or regression testing.

Time-Consuming

Exploratory testing is time-consuming because there are no predefined test cases provided, meaning there is no clear plan or estimation of when it can be completed.

Conclusion

Exploratory testing serves as a powerful approach to software testing, leveraging human intuition, creativity, and adaptability to uncover defects, and usability issues, and improve the overall quality of software applications. By encouraging testers to actively explore and experiment with the software, exploratory testing complements scripted testing and helps to ensure comprehensive test coverage. It offers the flexibility to adapt to changing requirements, provides valuable insights into the behavior of the application, and helps to identify critical issues early in the testing process. However, exploratory testing also has its limitations, including the reliance on tester skills, the lack of documentation, the effort required, and the potential for variability in results. To maximize its benefits, exploratory testing should be performed in conjunction with other testing approaches, leveraging the strengths of both scripted and exploratory testing methodologies. Finally, exploratory testing empowers software testers to unleash their creativity, critical thinking, and domain knowledge, resulting in improved software quality and a better user experience.

Author: Abdi Muflih Dzaky, Software Tester