Skip to main content

How to integrate TestNG Test Automation Results (Selenium/RestAssured/Appium) with TestRail?

 


To integrate TestRail with any automation suite, we can use the TestRail APIs with the following basic flow:

  • Create test cases in TestRail. 
  • Provide those unique Test IDs from TestRail to your TestNG tests.
  • Run automation:
    • Create a test suite in TestRail before test execution using the TestRail API.
    • And then post the run results according to test automation results using the TestRail API.
  1. Create test cases in TestRail:



       2. Creating a custom annotation and calling it as TestRails:






     3. After that, we associate our test cases with actual Test Rails ID’s like:




    4. These ids 1, 2, 3, and 4 are test case ID that we got from TestRail test cases, you can hover over your test case and see that ID in the URL (as shown below):






    5. Create a Test Run in TestRail through code,  as you can look at that we have not created any Test Run manually (see below):




Snapshot of TestRail before execution of automation suite:







6. Retrieve Test Case ID from Annotation:




7. Run the test cases and update Test Rail Status:





Let’s run the test cases and see if it is working:







As you can see that 2 test cases are passed and the other 2 are failed as expected based on the assertions we added, let’s see the status in TestRail:

Voila… Worked perfectly fine. (The test run is created; plus, test statuses are updated.)

Snapshot of TestRail after execution of automation suite:






And for the failed case, we can see the error message also as we added that comment in the code:





Points to remember:


There are 2 ways in which we can update our TestRail results after a test case execution:

    • Create a TestNG listener that will listen for test results and call the TestRail API to update it with any results it finds
    • Call the TestRail API inside the test method to update results (I have used this one for now)

There are 2 methods to update test results:

  • add_result
  • add_result_for_case


The difference between these two is that “add_result” takes the Test ID, "add_result_for_case" takes the Case ID and Run ID. We can use any method depending on the automation approach selected but "add_result_for_case" comes with an advantage over "add_result" i.e., while creating a test run as per the iteration, the test ID changes corresponding to the test run ID, but the test case IDs remains the same. This makes it easy to maintain the test case ID and run the automation suite repeatedly.



Do comment if you would like to have the entire code of this integration piece.


To Integrate Automated TestNG @Test with @DataProvider Results in TestRail, please check this blog:









Labels:

Comments

Post a Comment

Popular posts from this blog

Clipboardy: Node.js library

 Clipboardy is a popular Node.js library that provides a cross-platform solution for interacting with the system clipboard. It supports both synchronous and asynchronous operations, and it can be used to copy, paste, and read the clipboard contents. Features of Clipboardy: 1. Cross-platform compatibility: It works on macOS, Windows, Linux, OpenBSD, FreeBSD, Android with Termux, and modern browsers. 2. Synchronous and asynchronous operations: It provides both synchronous and asynchronous methods for clipboard operations, making it suitable for a variety of use cases. 3. Copy, paste, and read operations: It supports copying, pasting, and reading the clipboard contents, making it a versatile tool for interacting with the clipboard. 4. Promise-based API: It is an asynchronous method that returns promises, making it easy to handle asynchronous operations in a clean and concise way. 5. Easy to use: It has a simple and intuitive API that is easy to learn and use. How to Use Clipboardy: To...
Post a Comment
Read more

ARIA Snapshot in Playwright

  What is an ARIA Snapshot in Playwright? An  ARIA snapshot  in Playwright is a structured representation of a page’s  accessibility tree , which is used by assistive technologies (e.g., screen readers) to interpret the content of a web page. This snapshot helps verify if elements have the correct  roles, names, and properties  required for accessibility. Playwright provides the page.accessibility.snapshot() API to capture this accessibility tree at any given moment during test execution. How Does ARIA Work? ARIA ( Accessible Rich Internet Applications ) is a set of attributes that help improve accessibility by defining roles, states, and properties for elements that are not natively accessible. Example: In this case, the aria-label ensures that screen readers identify the button as “Submit Form.” How to Use ARIA Snapshots in Playwright? Playwright’s  accessibility.snapshot()   method retrieves the  accessible structure  of the page. Ex...
Post a Comment
Read more

Bruno vs Postman: Which API Client Should You Choose?

  As API testing becomes more central to modern software development, the tools we use to test, automate, and debug APIs can make a big difference. For years, Postman has been the go-to API client for developers and testers alike. But now, Bruno , a relatively new open-source API client, is making waves in the community. Let’s break down how Bruno compares to Postman and why you might consider switching or using both depending on your use case. ✨ What is Bruno? Bruno is an open-source, Git-friendly API client built for developers and testers who prefer simplicity, speed, and local-first development. It stores your API collections as plain text in your repo, making it easy to version, review, and collaborate on API definitions. 🌟 What is Postman? Postman is a full-fledged API platform that offers everything from API testing, documentation, and automation to mock servers and monitoring. It comes with a polished UI, robust integration, and support for collaborati...
Post a Comment
Read more