Skip to main content

Defining the scope of our End to End tests


We all should try that our End to End test should NOT considerably reiterate the test efforts of our Unit test and API test. Ideally, our end to end should be meant to confirm that user(s) can use our app in the way it's intended to do so, perform interactions with it without hitting any issues and always work when performing any E2E transaction(s). On the other hand, our Unit and API test should test and cover business logic. 

Many a time, we go overboard to reach 100% coverage, create an automated mess in terms of freezing a scope of our End to End tests and we might reach to a point:
- Where our test source code becomes even the same or in fact, more than the application codebase
- And/Or automation execution run time is likely to take the same time as it takes to write the test case, etc.

There's no silver bullet here and it's all about trying, failing and finally succeeding. One such approach can be:
Divide your test cases broadly into two major groups: Fat and Thin. Your Fat test cases should be what a max number of users performs repeatedly on your application. First target these and spend your max energy, effort and time on these because if these aren't working- as Russell Peters said- Somebody Gonna Get Hurt Real Bad.

Then only jump to Thin cases. What are Thin cases? These are those cases where users use our application mostly in an unexpected and unforeseen way but again as these can create issues in your app these are also important but obviously, these take a second priority. 

Based on the execution test time allowed (on our final test cycle) and regression impact, QE can recommend running both Fat and Thin tests (recommended) OR only Fat ones on the last cycle. 

P.S. As part of our daily CI/CD test automation run, we should try to run both Fat and Thin tests.

Labels

Labels:

Comments

Post a Comment

Popular posts from this blog

The use of Verbose attribute in testNG or POM.xml (maven-surefire-plugin)

At times, we see some weird behavior in your testNG execution and feel that the information displayed is insufficient and would like to see more details. At other times, the output on the console is too verbose and we may want to only see the errors. This is where a verbose attribute can help you- it is used to define the amount of logging to be performed on the console. The verbosity level is 0 to 10, where 10 is most detailed. Once you set it to 10, you'll see that console output will contain information regarding the tests, methods, and listeners, etc. <suite name="Suite" thread-count="5" verbose="10"> Note* You can specify -1 and this will put TestNG in debug mode. The default level is 0. Alternatively, you can set the verbose level through attribute in "maven-surefire-plugin" in pom.xml, as shown in the image. #testNG #automationTesting #verbose # #testAutomation
Post a Comment
Read more

How to Unzip files in Selenium (Java)?

1) Using Java (Lengthy way) : Create a utility and use it:>> import java.io.BufferedOutputStream; import org.openqa.selenium.io.Zip; import java.io.File; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.IOException; import java.util.zip.ZipEntry; import java.util.zip.ZipInputStream;   public class UnzipUtil {     private static final int BUFFER_SIZE = 4096;     public void unzip (String zipFilePath, String destDirectory) throws IOException {         File destDir = new File(destDirectory);         if (!destDir.exists()) {             destDir.mkdir();         }         ZipInputStream zipIn = new ZipInputStream(new FileInputStream(zipFilePath));         ZipEntry entry = zipIn.getNextEntry();         // to iterates over entries in the zip folder         while (en...
Post a Comment
Read more

Stop Overengineering: Why Test IDs Beat AI-Powered Locator Intelligence for UI Automation

  We have all read the blogs. We have all seen the charts showing how Generative AI can "revolutionize" test automation by magically resolving locators, self-healing broken selectors, and interpreting UI changes on the fly. There are many articles that paints a compelling picture of a future where tests maintain themselves. Cool story. But let’s take a step back. Why are we bending over backward to make tests smart enough to deal with ever-changing DOMs when there's a simpler, far more sustainable answer staring us in the face? -             Just use Test IDs. That’s it. That’s the post. But since blogs are supposed to be more than one sentence, let’s unpack this a bit. 1. Test IDs Never Lie (or Change) Good automation is about reliability and stability. Test IDs—like data-testid ="submit-button"—are predictable. They don’t break when a developer changes the CSS class, updates the layout, or renames an element. You know...
Post a Comment
Read more