Skip to main content

False-Positive & False-Negative in automation


False-Negative is a case where your test result comes out to be positive, but in actuality, an issue is present in the system and the functionality is not working as it should. And Vice-versa is False Positive, which is a case where the test execution results show an error, even though everything is working as intended. Fundamentally, both these have always posed a challenge for web automated testing but I guess it’s ok to say that a False-negative is more hurting than a False-Positive, as the former creates a false sense of security (a lie) and would definitely end up costing much more to us.

I agree that a False-Positive too consume a lot of our's time and worth. On average 70% of automated test case failures are False-Positives due to which testers spend an average of 1/3rd of their time analyzing, correcting and reporting results that actually should NOT need any attention at all. In fact, with CI/CD implementation, running automated tests every night or after every commit to the master branch, time wasted by testers to triage these false positives easily outweighs the quicker test results provided by our automation suite.

Do not trust on every tool that you use blindly, a few ways through which we can handle False-Positives/False-Negatives:

  • Put solid assertions, cover both +ve as well as -ve assertions. While the creation of your tests, make sure your tests fail when they should.
  • Revisit your tests regularly. Peer-Review them regularly, not only at the time of the creation of your tests suite.
  • Remove tests from your suite if they no longer serve a purpose. I know it's debatable.
  • Review your Test report properly.
  • Go for both manual and automation testing, so that you get more confidence that there are no False-Positives/False-Negatives in your system.
  • Dynamic synchronization of objects.
  • Use correct test data and revisit it regularly. Don't always go with independent or pre-set test data- your application test data should be in line with your test strategy.


What best practices do you follow to prevent and detect false positives and false negatives in your test automation?


Labels:

Comments

Post a Comment

Popular posts from this blog

ARIA Snapshot in Playwright

  What is an ARIA Snapshot in Playwright? An  ARIA snapshot  in Playwright is a structured representation of a page’s  accessibility tree , which is used by assistive technologies (e.g., screen readers) to interpret the content of a web page. This snapshot helps verify if elements have the correct  roles, names, and properties  required for accessibility. Playwright provides the page.accessibility.snapshot() API to capture this accessibility tree at any given moment during test execution. How Does ARIA Work? ARIA ( Accessible Rich Internet Applications ) is a set of attributes that help improve accessibility by defining roles, states, and properties for elements that are not natively accessible. Example: In this case, the aria-label ensures that screen readers identify the button as “Submit Form.” How to Use ARIA Snapshots in Playwright? Playwright’s  accessibility.snapshot()   method retrieves the  accessible structure  of the page. Ex...
Post a Comment
Read more

Bruno vs Postman: Which API Client Should You Choose?

  As API testing becomes more central to modern software development, the tools we use to test, automate, and debug APIs can make a big difference. For years, Postman has been the go-to API client for developers and testers alike. But now, Bruno , a relatively new open-source API client, is making waves in the community. Let’s break down how Bruno compares to Postman and why you might consider switching or using both depending on your use case. ✨ What is Bruno? Bruno is an open-source, Git-friendly API client built for developers and testers who prefer simplicity, speed, and local-first development. It stores your API collections as plain text in your repo, making it easy to version, review, and collaborate on API definitions. 🌟 What is Postman? Postman is a full-fledged API platform that offers everything from API testing, documentation, and automation to mock servers and monitoring. It comes with a polished UI, robust integration, and support for collaborati...
Post a Comment
Read more

🔧 Self-Healing Selenium Automation with Java — A Smarter Way to Handle Broken Locators

  How to build smarter, more resilient automated tests? We’ve all been there — our Selenium test cases start failing because of minor UI changes like updated element IDs, renamed classes, or even reordered elements. It’s frustrating, time-consuming, and often the most dreaded part of maintaining automated tests. But what if your automation could heal itself? 💡 What is Self-Healing Automation? Self-healing automation  refers to the capability of a test automation framework to recover from minor UI changes by automatically trying alternative locators when the primary one fails. It’s like giving your test scripts a survival instinct. 🔨 🛠️ Implementation in Java + Selenium: Step by Step Step 1: Create a Self-Healing Wrapper We start by creating a custom class called SelfHealingDriver. This class wraps the standard WebDriver and handles locator failures gracefully. public   class   SelfHealingDriver { private   WebDriver driver ; public   SelfHealingDri...
Post a Comment
Read more