Automation testing is not just about executing test cases—it's also about presenting the results in a clear and meaningful way. While TestNG provides basic HTML reports, enterprise teams often use Allure Reports because they offer interactive dashboards, screenshots, execution history, attachments, and detailed failure analysis.
In this guide, you'll learn how to integrate Allure Reports with Playwright using Java and TestNG.
Why Use Allure Reports?
Allure Reports provide:
- Interactive HTML reports
- Test execution history
- Test categorization
- Screenshots for failed tests
- Video attachments
- Environment information
- Timeline view
- Easy CI/CD integration
Compared to standard TestNG reports, Allure makes debugging faster and reporting more professional.
Prerequisites
Before starting, ensure you have:
- Java 17 or later
- Maven installed
- Playwright project
- TestNG framework
- Browser binaries installed
Step 1: Add Maven Dependencies
Add the following dependencies to your pom.xml.
<dependencies>
<dependency>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-testng</artifactId>
<version>2.30.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.playwright</groupId>
<artifactId>playwright</artifactId>
<version>1.55.0</version>
</dependency>
</dependencies>Refresh your Maven project after saving the file.
Step 2: Configure Maven Surefire Plugin
Add the Surefire plugin to enable TestNG execution.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.5.3</version>
</plugin>
</plugins>
</build>Step 3: Execute the Tests
Run your TestNG suite.
mvn clean testDuring execution, Allure stores raw result files inside:
allure-results/Each test generates JSON files containing execution details.
Step 4: Generate the Report
Once execution is complete, generate the HTML report.
allure serve allure-resultsThis command:
- Builds the report
- Starts a local server
- Opens the report automatically in your browser
Understanding the Report Dashboard
The Allure dashboard provides valuable insights.
You'll find:
- Total tests
- Passed tests
- Failed tests
- Skipped tests
- Duration
- Success rate
This overview helps teams quickly assess test health.
Test Details
Clicking a test reveals:
- Execution steps
- Attachments
- Error messages
- Stack traces
- Screenshots
- Execution time
This detailed view speeds up root cause analysis.
Adding Screenshots to Allure
Capturing screenshots on failures greatly improves debugging.
Example helper method:
@Attachment(
value = "Failure Screenshot",
type = "image/png"
)
public byte[] attachScreenshot(Page page){
return page.screenshot();
}When called during a failed test, the screenshot is embedded directly in the report.
Attaching Text Logs
You can also include custom logs.
@Attachment(
value = "Execution Log",
type = "text/plain"
)
public String attachLog(String message){
return message;
}This is useful for recording API responses, SQL queries, or business events.
Attaching Videos
If Playwright records execution videos, attach them to the report.
@Attachment(
value = "Execution Video",
type = "video/webm"
)
public byte[] attachVideo(Path path)
throws IOException{
return Files.readAllBytes(path);
}Videos are invaluable when debugging intermittent failures.
Adding Test Descriptions
Allure supports annotations that make reports more readable.
@Description("Verify successful login using valid credentials")
@Test
public void loginTest(){
}Descriptions appear directly in the report.
Categorizing Tests
Use features and stories to organize tests.
@Feature("Authentication")
@Story("Login")
@Test
public void loginTest(){
}Reports become easier to navigate, especially in large projects.
Organizing the Project
Recommended folder structure:
project
├── allure-results
├── allure-report
├── screenshots
├── videos
├── src
├── pom.xml
└── testng.xmlKeeping artifacts separate improves project organization.
Best Practices
Capture Screenshots Only on Failure
Avoid capturing screenshots for every successful test to reduce storage usage.
Add Meaningful Descriptions
Use annotations to explain the purpose of each test.
Attach Logs for Failed Scenarios
Include:
- API responses
- Request payloads
- Browser console logs
- Custom debug messages
These details make troubleshooting much easier.
Store Environment Information
Create an environment.properties file containing details such as:
- Browser
- Operating system
- Java version
- Playwright version
- Test environment
This information appears in the report and helps reproduce issues.
Archive Reports in CI/CD
When running tests in Jenkins, GitHub Actions, or Azure DevOps, archive the generated Allure reports as build artifacts. This allows team members to review historical execution results.
Common Interview Questions
Why is Allure preferred over the default TestNG report?
Because it provides interactive reports with screenshots, attachments, execution history, and a richer user experience.
How do you generate an Allure report?
Execute the tests and then run:
allure serve allure-resultsCan Allure attach screenshots automatically?
Yes. By using the @Attachment annotation and capturing screenshots during test failures.
What types of files can be attached to Allure reports?
- Screenshots
- Videos
- Text logs
- JSON responses
- XML files
- PDFs
- Any file that helps explain the test outcome
Conclusion
A good reporting solution is essential for every automation framework.
Allure Reports transform raw test execution data into an interactive, easy-to-understand dashboard that helps testers, developers, and managers quickly identify issues.
In this guide, you learned how to:
- Integrate Allure with Playwright
- Configure Maven
- Generate reports
- Attach screenshots, videos, and logs
- Organize test results
- Follow enterprise reporting best practices
By combining Playwright, TestNG, and Allure Reports, you can build a professional automation framework that is scalable, maintainable, and ready for enterprise projects.
No comments:
Post a Comment