TestNG Tutorial: Annotations, Assertions and testng.xml Explained

If you have written a few Selenium scripts with a main method, you already know the pain: no clean setup and teardown, no report, no way to run only the smoke tests, and one failure stops everything. This TestNG tutorial shows how the framework solves exactly these problems, which is why it sits underneath most Java automation frameworks in Indian companies.

We will walk through a working setup with Maven, the annotations you will use daily, assertions, data-driven tests, groups, and the testng.xml file that ties a suite together. Every example runs on Java 17 with TestNG 7. If you are new to Selenium itself, read our Selenium WebDriver tutorial first and then come back.

Key takeaways

• TestNG is a test runner and framework for Java that adds annotations, assertions, grouping, parameters, parallel runs and reports.
• Annotations follow a fixed order: suite, test, class, method. Getting this order right prevents most setup bugs.
• Use hard assertions for checks that make the rest of the test meaningless, and soft assertions when you want to collect several failures.
• @DataProvider and testng.xml parameters make tests data-driven without duplicating code.
• testng.xml controls which classes, groups and parameters run, and how many threads to use.

What is TestNG and why testers use it

TestNG (“Next Generation”) is an open-source testing framework for Java, inspired by JUnit but designed with broader testing needs in mind. It does not drive the browser; Selenium does that. TestNG decides what runs, in which order, with which data, and how results are reported.

Teams pick TestNG because it offers:

• Rich annotations for setup and teardown at several levels.
• Built-in grouping, so you can run “smoke” or “regression” from the same codebase.
• Data providers for running one test with many data sets.
• Dependencies between tests when you really need them.
• Parallel execution configured in XML.
• Listeners for custom logging, screenshots and retries.

JUnit 5 now covers much of this too, and some teams prefer it. In job descriptions for Selenium with Java roles in India, though, TestNG still appears very often, so it is worth learning well.

Setting up TestNG with Maven

Create a Maven project and add TestNG and Selenium to pom.xml. TestNG 7.10 requires Java 11 or later.

<dependencies>
  <dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version>7.10.2</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>4.25.0</version>
  </dependency>
</dependencies>

To run the suite file from Maven, configure the Surefire plugin:

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>3.2.5</version>
      <configuration>
        <suiteXmlFiles>
          <suiteXmlFile>testng.xml</suiteXmlFile>
        </suiteXmlFiles>
      </configuration>
    </plugin>
  </plugins>
</build>

In IntelliJ IDEA, TestNG support is built in. In Eclipse, install the TestNG plugin from the marketplace so you can right-click and run tests.

TestNG tutorial: your first test

Here is a small test class that opens a page and checks its title. Selenium Manager, included with Selenium 4, downloads the matching browser driver automatically.

package tests;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.Test;

public class HomePageTest {

    private WebDriver driver;

    @BeforeMethod
    public void setUp() {
        driver = new ChromeDriver();
        driver.get("https://example.com");
    }

    @Test
    public void titleShouldBeCorrect() {
        Assert.assertEquals(driver.getTitle(), "Example Domain", "Page title mismatch");
    }

    @AfterMethod
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Note the argument order in TestNG’s assertEquals: actual value first, expected value second. JUnit uses the opposite order, and mixing them up produces confusing failure messages.

TestNG annotations and execution order

The annotations you will use most

Execution order

The order is always: @BeforeSuite, @BeforeTest, @BeforeClass, @BeforeMethod, @Test, @AfterMethod, @AfterClass, @AfterTest, @AfterSuite. A common interview trick is to confuse @BeforeTest with @BeforeMethod. Remember that “test” in @BeforeTest refers to the <test> tag in testng.xml, not to each test method.

Useful @Test attributes

• priority = 1 sets order within a class; lower numbers run first.
• groups = {"smoke"} tags a test for selective runs.
• dependsOnMethods = "login" skips the test if login fails.
• enabled = false temporarily disables a test.
• timeOut = 5000 fails a test that runs longer than five seconds.
• expectedExceptions = IllegalArgumentException.class passes only if that exception is thrown.
• invocationCount = 3 runs the same test three times.

Use dependsOnMethods sparingly. Independent tests are easier to run in parallel and easier to debug.

Assertions in TestNG: hard vs soft

Hard assertions

Methods on org.testng.Assert stop the test at the first failure. Use them when a failure makes later steps pointless, such as a failed login.

Soft assertions

SoftAssert records failures and keeps going. You must call assertAll() at the end, or failures are silently ignored.

import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;

public class ProfileTest {

    @Test
    public void profileFieldsShouldMatch() {
        SoftAssert soft = new SoftAssert();
        String name = "Asha Kulkarni";
        String city = "Pune";

        soft.assertEquals(name, "Asha Kulkarni", "Name mismatch");
        soft.assertEquals(city, "Pune", "City mismatch");
        soft.assertTrue(name.length() > 3, "Name too short");

        soft.assertAll();
    }
}

Create a new SoftAssert inside each test method. Sharing one instance across tests mixes failures from different tests together.

Data-driven tests with @DataProvider

A data provider returns rows of data; TestNG runs the test once per row.

import org.testng.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;

public class LoginValidationTest {

    @DataProvider(name = "invalidEmails")
    public Object[][] invalidEmails() {
        return new Object[][] {
            {"plainaddress"},
            {"@no-local-part.com"},
            {"user@"},
            {"user@domain..com"}
        };
    }

    @Test(dataProvider = "invalidEmails")
    public void shouldRejectInvalidEmail(String email) {
        boolean valid = email.matches("^[\\w.+-]+@[\\w-]+(\\.[\\w-]+)+$");
        Assert.assertFalse(valid, "Expected invalid: " + email);
    }
}

In real projects, the data provider often reads from a CSV, JSON or Excel file instead of hard-coded values. Keep the reading logic in a utility class so tests stay short.

Understanding testng.xml

testng.xml is the suite file. It decides which classes or packages run, which groups are included, what parameters are passed and how parallel execution works.

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression" parallel="classes" thread-count="3">
  <test name="Chrome tests">
    <parameter name="browser" value="chrome"/>
    <groups>
      <run>
        <include name="smoke"/>
      </run>
    </groups>
    <classes>
      <class name="tests.HomePageTest"/>
      <class name="tests.LoginValidationTest"/>
    </classes>
  </test>
</suite>

Read a parameter in a setup method:

@Parameters("browser")
@BeforeMethod
public void setUp(String browser) {
    System.out.println("Running on " + browser);
}

Key points to remember:

• parallel can be methods, classes, tests or instances. Parallel runs require a thread-safe driver, usually a ThreadLocal<WebDriver>.
• You can have several <test> blocks, for example one per browser.
• Groups can be included or excluded per <test>.
• Run it from the IDE by right-clicking the file, or from the command line with mvn test.

Listeners, retries and reports

Listeners let you react to test events. A common use is taking a screenshot on failure.

import org.testng.ITestListener;
import org.testng.ITestResult;

public class FailureListener implements ITestListener {
    @Override
    public void onTestFailure(ITestResult result) {
        System.out.println("Failed: " + result.getName() + " - " + result.getThrowable());
    }
}

Register it with @Listeners(FailureListener.class) on a class, or in testng.xml under a <listeners> tag. For flaky infrastructure, IRetryAnalyzer can re-run a failed test a limited number of times, but treat retries as a safety net, not a fix.

TestNG writes default HTML and XML reports to test-output (or target/surefire-reports when run through Maven). Many teams add Allure or Extent Reports for richer output.

Organising a real TestNG project

Once you have more than a handful of classes, structure matters as much as annotations. A layout that scales well:

• A BaseTest class holds @BeforeMethod and @AfterMethod for browser setup and teardown, and every test class extends it.
• Page classes live in their own package and contain locators and actions only, with no assertions.
• Data providers sit in a separate class and are referenced with dataProviderClass, so several test classes can share them.
• Listeners live in a listeners package and are registered once in testng.xml.
• One suite file per purpose, for example smoke.xml, regression.xml and cross-browser.xml, each selecting different groups or parameters.

Running a specific suite from the command line then looks like this:

mvn test -Dsurefire.suiteXmlFiles=smoke.xml

In CI tools such as Jenkins or GitHub Actions, the same command runs on every build, and the Surefire XML results in target/surefire-reports let the CI server show pass and fail counts per test. Getting this structure right early saves a painful refactor when the suite grows from 20 tests to 500.

Common mistakes beginners make

• Forgetting assertAll() with soft assertions.
• Sharing a single WebDriver field across parallel tests.
• Overusing priority and dependsOnMethods to force an order instead of making tests independent.
• Mixing up @BeforeTest and @BeforeMethod.
• Hard-coding environment URLs inside tests instead of using parameters or a config file.

Avoiding them is the real goal of any TestNG tutorial, because they cause most of the confusing failures teams see. They come up often in interviews too. Our list of Selenium interview questions includes several TestNG follow-ups worth practising.

FAQs

Is TestNG better than JUnit?

Neither is strictly better. TestNG has long been popular in Selenium projects for its XML suites, groups and data providers, while JUnit 5 is the default in many developer teams and offers similar features through extensions. Learn the one your target jobs mention most, and the other at a basic level.

What is the use of testng.xml?

It defines the suite: which classes, packages or groups run, what parameters are passed, and how parallel execution is configured. It lets you run different sets of tests without changing code.

How do I run TestNG tests from the command line?

With Maven and the Surefire plugin configured, run mvn test. You can also pass a suite file with -Dsurefire.suiteXmlFiles=testng.xml if it is not set in the POM.

What is the difference between @BeforeTest and @BeforeMethod?

@BeforeTest runs once before each <test> block defined in testng.xml. @BeforeMethod runs before every individual test method. Use @BeforeMethod to open a fresh browser for each test.

Can TestNG run tests in parallel?

Yes. Set the parallel and thread-count attributes in testng.xml. Make sure your driver is thread-safe, usually by storing it in a ThreadLocal.

Conclusion

Take one small Selenium script you already have, convert it into a TestNG class with @BeforeMethod, @AfterMethod and two assertions, then run it through a testng.xml file with a smoke group. Once that works, add a data provider. The official TestNG documentation is the best reference as you go, and these skills are a solid step on the path to becoming an SDET.