Exploratory Testing In Agile Software Testing

Exploratory Testing In Agile Software Testing: Sometimes, testers don’t get the requirement document, but they must finish all testing activities within the limited time frame. Within this strict timeline, they have to explore, learn the application, do test planning, test the application, and present a detailed test analysis. So, how can all this be achieved within such a short span?

Hence the need for exploratory testing. Exploratory testing follows the approach of minimum planning and maximum test execution, which involves logical and critical thinking. There is neither a test case to read and execute nor automated test scripts to run as designed. Exploratory testing is one of the strongest ways to understand and test the application with logic. This enhances the quality to a higher extent and can be combined with other testing practices simultaneously.

In an article, James Bach wrote.“Exploratory software testing is a powerful and fun approach to testing. In some situations, it can be orders of magnitude more productive than scripted testing”.

One way of putting it is exploratory testing is a documented form of random testing. This is the opposite of the formal/scripted testing process, which covers all phases of software testing. In Exploratory testing, testers will not follow any particular lifecycle or approach. They will validate the application by performing various tests on the application in their way, become familiar with it, and capture any errors found during the testing process.

Exploratory Testing Definition

• Exploratory Testing is a testing approach that allows you to apply your ability and skill as a tester in a powerful way.”
• Testers have to understand the application first by exploring the application. Based on this understanding, they should come up with the test scenarios after that start actual testing of the application.

Key tips to remember in Exploratory testing techniques:

• The preparation of test scenarios validates the software’s stability.
• Exhaustively testing the software based on the identified requirements.
• Find out the requirements as well as the functionality of the software application.
• Find out the limitation of the software application.
• Identify the scope of the project.

Exploratory Testing In Agile

Exploratory testing is sometimes referred to as ad hoc testing or monkey testing. All these terms focus on the same but are not exactly the same.

• Monkey testing is a random way of testing the application. Once the issue is observed, analysis of the issue for what could have caused it comes into the picture. This is sometimes called Random Testing. While performing monkey testing, the test might detect a handful of anomalies from the application. But are you sure that the application is tested for user scenarios? The answer is NO (So it can’t be exactly termed as exploratory testing). Even though it is random testing, bit does follow some systematic approach while a tester performs it.
• Ad Hoc testing is testing the application randomly but with complete knowledge of the application and/or domain. Here tester focuses on every scenario being tested and how it is being tested. Also, alternate scenarios are focused on and tried out in case of issues. But again, this does not have any systematic approach as such. Experienced testing professionals generally do it, requiring a good imagination and analytical skills, but this is not the learning phase.

Exploratory Testing generally means “Testing application without Requirement Specifications”. But it is more than that:

• Skilled testers perform exploratory testing once scripted test execution is done, and brainstorming is done to find more scenarios.
• If the time to test an application is less, then exploratory is the best way to achieve test coverage. Normally we have seen the sprint cycles have even reduced to 1 week, so in these cases, it becomes difficult for the tester to write test cases and execute them.
• Exploratory means NOT to test with scripted test cases, but you should be aware of full requirements and think beyond it.
• Any tester testing without referring to scripted test cases is performing exploratory testing.
• Corner areas can be reached through efficient exploratory testing.

“Discovering the unexpected is more important than confirming the known.“— George E. P. Box.  

Advantages of Exploratory Testing

Some of the striking advantages of exploratory testing are:

• Testers get familiar with the application and simultaneously check whether the application under test behaves as expected. In fact, the tester is doing the same thing as traditional testers do, but without documentation.
• The tester can find the bug quickly as compared to traditional testing. Results have shown that the rate at which new bugs are found is much higher for exploratory testing than scripted testing.
• Many times, it’s not mandatory to follow the traditional approach and prepare test cases for scenarios that do not have much business impact. For such scenarios, exploratory testing is recommended. Also, exploratory testing is recommended when the software versions are early, and the tester must be familiar with the application.
• Since much logical thinking is required for Exploratory testing, the outcome is the best quality, while in cases of scripted testing, the objective is to pass/fail the test without brainstorming on test scenarios.
• Exploratory can be done at any test level and type. Either performance, security, integration, etc. In general, it can be included in all testing practices.
• Uncommon integration issues can be uncovered by rigorous exploratory testing, which could detect more possible ways to break the application.
• It doesn’t require preparation for testing as we don’t have documents.
• In this type of testing, time saves due to all tasks simultaneously, like Testing, Designing test scenarios, and executing test scenarios.
• The tester can report many issues due to incomplete requirements or missing required documents.

However, exploratory testing does not guarantee the quality of the application, but it is advantageous, especially during time crunch or shoe-string budgets.

Disadvantages of Exploratory Testing

• Not able to showcase the efforts inputted for exploratory testing since the most activity is not documented.
• Not possible to showcase the quality of exploratory testing if bugs weren’t found.
• The outcome of exploratory usually comes up with some unanswered queries which need to be clarified by clients. Taking assumptions most of them degrade the objective & effectiveness of Exploratory Testing.
• Exploratory testing is only effective if done for some hour’s duration. It is never a day-long activity.
• Mostly small tight schedules projects perform exploratory in that test case expected results for exploratory remains unreviewed.
• Without much critical thinking during exploratory testing, the outcome could be much more disappointing than scripted testing.
• Finalizing the test coverage is challenging, as there is no formal mapping between the tests performed and requirements. Until the end of User Acceptance Testing, the results of exploratory testing are not much consideration for quality.
• Since this is an unplanned activity, convincing customers about the expected efforts and time is difficult.
• Sometimes, this testing is taken lightly, and areas are left untested. Also, chances are there to leave the defects as no documentation is involved in testing.
• This does not help in root cause analysis in case of missed defects in production.
• Few issues cannot be the catch in this type of testing.
• There is a review of test planning & designing of test cases/scenarios while testing may cause issues.
• Testers must remember the scenario he is executing because if any bug is found, the tester should report a bug with proper steps to reproduce the Difficulty in performing the exact manner, especially for newly found bugs.

Challenges faced in Exploratory Testing

As in any other testing type, even exploratory testing has challenges:

• The entire application and the purpose of each and every feature have to be learned in a short span of time.
• The term “unplanned activity” raises many questions about the effort, time, and efficiency that the tester has put in. Difficult to convince customers and management about the actual accounts.
• There will be no proof to show the effort put in unless defects are logged. In this case, it is challenging to defend the actual tests performed.
• Since test cases and scripts are not involved in exploratory testing, the tester must continuously think about the possible tests to execute, which may become stressful.
• Difficult to reproduce the defects in case of corner scenarios
• Test Coverage is difficult to prove if it is clubbed with system/regression testing, as their results show most of the coverage, and exploratory has little to show the coverage.
• Difficult to record each and every test performed.
• Tedious when exploratory testing results have to be reported.

Chartered Exploratory Testing

There are many ways to perform Exploratory testing, but one of the most commonly used methods is Chartered exploratory testing. In this method, the tester must make one charter to guide how to test within a defined timeline. So first, we need to prepare the Charter, and then the time box should be defined so as to finish the task within the defined timeline.

Mission

• Defines the objective we need to achieve from exploratory tests.
• What features do we need to test, and how to test
• Testers may focus on specific complex functionalities for a particular application in one session.
• Testers may choose one of the system workflows and test it in a particular session.

Charter

• What we want to achieve
• What tactics to use
• What risks are involved
• What kinds of defects to look for
• Any high-level test plan
• Who will be involved? Getting the test lead involved or performing pair-to-pair testing would be good.

Session

• Generally varies from 2-3 hours.
• Breaks while performing exploratory testing aren’t recommended.
• Don’t break current scenarios in-between if any other scenario is clicked.

Session Report

• Testers can prepare test notes while exploring the application and prepare notes for their own learning.
• If possible, notes on all the scenarios covered.
• Note if clarification is required with the client for some scenarios.

Bugs

• List of bugs identified.
• Raising them on Defect Management Tool with proper classification

Discussion

• Schedule a meeting with the manager to provide information regarding the different activities.
• Prepare a report and share it with the team.
• Keep documents prepared in Session Report for future reference.

Even though documentation is not mandatory for exploratory testing, creating a one-page document at the end of exploratory testing is advisable.  This documentation can be submitted to the manager to showcase the testing status and the application’s current state.

Where To Use Exploratory Testing?

• Exploratory testing may be useful when testing needs to be completed without any proper requirement document.
• It is also recommended when the time available for testing is slim.
• When the project has budget-related constraints.
• If the modules are simple and don’t require elaborate test documentation in the form of test cases.
• You need to add Critical thinking to find more defects.

Misconceptions Regarding Exploratory Testing

Lots of misconceptions are there regarding exploratory testing. Some of them are as follows:

• Unstructured: Many people think it is an incorrect form of testing. It is structured and only done by testers who have ample product knowledge.
• No need to plan or document tests: Many people think there is no need to plan or document tests. But it’s a complete myth. The plan is mentioned in the charter though it is very short and may be contained in a paragraph.
• It is new and fast: Many people believe that Exploratory testing is new and can help in test completion at a fast pace. But the fact is, testers have been using exploratory testing for a very long time. Even though it is a swift alternative, it is a time-bound affair and can take up quite a lot of time, depending on the complexity of the flow or module under test.
• It is more effective at finding bugs: It is false. Traditional or scripting testing is also quite effective and maybe even more effective at finding bugs than exploratory testing.
• No need to do scripted testing: It is also not true. Scripted testing helps in sequentially covering all elements of the software to keep track of the progress and be confident of the quality of the software. Scripted or traditional testing needs to be applied along with Exploratory testing to get the best effect.

How to Be a Good Exploratory Tester

• Be aware of the domain you are performing testing.
• Gone through all the scripted test cases so you can think of scenarios beyond that.
• Logical thinking is the key. Don’t confuse exploratory with monkey testing.
• Get an idea, and talk to the client and developers to be aware of the app’s complete functionality.
• Don’t follow assumptions while doing exploratory; note them down for clarifications.
• Plan your tests i.e., objectives you want to achieve with the current exploratory.
• Don’t perform regression testing, and exploratory tests have the objective to finding issues where critical thinking is required.
• Don’t depend on exploratory testing tools; this type of testing is best with individuals until AI testing tools are available for testing.

Conclusion

Most of the experienced testers, while performing exploratory testing, leverage their product knowledge to test the application intuitively. That is why the experienced tester does exploratory testing.

For this reason, fresh testers should follow traditional scripted testing techniques and go through the test documentation while testing the application.

Overall we can say that exploratory testing is an intelligent way of doing testing, especially when the need of the hour is to deliver quality applications within a limited time frame. Also, it helps improve the application’s design in case of integration issues that end-users rarely encounter.

Even though documentation is not mandatory, good exploratory tests are planned, engaging, and creative in nature.

This article is the outcome of my thoughts added to the exploratory testing we have been performing in daily testing. If you want to read more about Exploratory, please follow the articles mentioned in the reference section.