...
Playwright
...
Playwright comes with several built-in reporters that help you visualize, log, and analyze test results. Here's a brief overview:
...
List Reporter (list):
This is the default reporter.
It provides a simple, human-readable output in the terminal, listing each test as it runs.
Shows test results (pass/fail) with clear indicators.
Dot Reporter (dot):
...
is
...
Good for situations where you want very concise feedback.
...
Line Reporter (line):
Similar to the
dotreporter, but each test is displayed on a single line.Provides more information than
dotbut still remains compact.
...
JSON Reporter (json):
Outputs test results in JSON format.
Useful for processing test results programmatically or for integrating with other tools.
...
JUnit Reporter (junit):
Generates an XML report in the JUnit format.
This is often used in CI/CD pipelines for integrating with test reporting tools.
...
HTML Reporter (html):
Generates a detailed HTML report.
Includes test results, screenshots, and videos (if configured).
Allows you to drill down into each test case and step.
...
Third-party reporters are community-contributed plugins that extend Playwright's reporting capabilities.
These reporters often integrate with popular testing and CI/CD tools, offering additional features or customizations that aren't available in Playwright's built-in reporters.
For more details: https://playwright.dev/docs/test-reporters#introduction
Custom Reporter using Playwright API
Playwright Offers API to write custom test reports. A custom reporter is a class that implements specific methods to handle events during the test run. Playwright Reporter API can be implemented and the native reporter methods of API can overridden based on needs. Below is an example:
PlaywrightCustomReporter.js
| Code Block | ||
|---|---|---|
| ||
class MyCustomReporter {
onBegin(config, suite) {
console.log(`Starting the test run with ${suite.allTests().length} tests`);
}
onSuiteBegin(suite) {
console.log(`Starting suite: ${suite.title}`);
}
onSuiteEnd(suite) {
console.log(`Finished suite: ${suite.title}`);
}
onTestBegin(test) {
console.log(`Starting test: ${test.title}`);
}
onTestEnd(test, result) {
console.log(`Finished test: ${test.title} with status ${result.status}`);
if (result.error) {
console.error(`Error: ${result.error.message}`);
}
}
onTestFailure(test, result) {
console.error(`Test failed: ${test.title}`);
console.error(`Error: ${result.error.message}`);
}
onTestSuccess(test, result) {
console.log(`Test passed: ${test.title}`);
}
onTestTimeout(test) {
console.warn(`Test timed out: ${test.title}`);
}
// Called when a test's retry count is exhausted
onTestRetry(test) {
console.warn(`Test retried: ${test.title}`);
}
onEnd(result) {
console.log(`Test run finished with status ${result.status}`);
console.log(`Total tests: ${result.totalTests}`);
console.log(`Passed: ${result.passedTests}`);
console.log(`Failed: ${result.failedTests}`);
}
onStdOut(data) {
process.stdout.write(data);
}
onStdErr(data) {
process.stderr.write(data);
}
onStepBegin(test, step) {
console.log('Step Begin:');
console.log('Test:', test);
console.log('Step:', step);
}
onStepEnd(test, step, result) {
console.log('Step End:');
console.log('Test:', test);
console.log('Step:', step);
console.log('Result:', result);
}
}
export default MyCustomReporter;
|
This code defines a custom reporter class MyCustomReporter that implements various methods to track and log the progress of a test run. It's designed to be used with a testing framework, likely Playwright, where it provides detailed logs at different stages of the test execution process.
Here’s a brief explanation of each method:
onBegin(config, suite): Logs the start of the test run, including the number of tests in the suite.onSuiteBegin(suite): Logs the start of a test suite.onSuiteEnd(suite): Logs the end of a test suite.onTestBegin(test): Logs the start of an individual test.onTestEnd(test, result): Logs the end of an individual test, including its status and any errors if they occurred.onTestFailure(test, result): Logs detailed information about a failed test.onTestSuccess(test, result): Logs a message when a test passes.onTestTimeout(test): Logs when a test times out.onTestRetry(test): Logs when a test is retried.onEnd(result): Logs the summary of the test run, including the status and counts of total, passed, and failed tests.onStdOut(data): Writes stdout output to the console.onStdErr(data): Writes stderr output to the console.onStepBegin(test, step): Logs the start of a test step, along with the test and step details.
| Code Block | ||
|---|---|---|
| ||
{
"test": {
"title": "basic test",
"_only": false,
"_requireFile": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"results": [
{
"retry": 0,
"parallelIndex": 0,
"workerIndex": 0,
"duration": 2662,
"startTime": "2024-08-21T14:36:32.149Z",
"stdout": [],
"stderr": [],
"attachments": [],
"status": "passed",
"steps": [
{
"title": "Before Hooks",
"category": "hook",
"startTime": "2024-08-21T14:36:32.156Z",
"duration": 1519,
"steps": [
{
"title": "fixture: browser",
"category": "fixture",
"startTime": "2024-08-21T14:36:32.170Z",
"duration": 304,
"steps": [
{
"title": "browserType.launch",
"category": "pw:api",
"startTime": "2024-08-21T14:36:32.176Z",
"duration": 298,
"steps": []
}
]
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:36:32.476Z",
"duration": 20,
"steps": [
{
"title": "browser.newContext",
"category": "pw:api",
"startTime": "2024-08-21T14:36:32.480Z",
"duration": 12,
"steps": []
}
]
},
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:36:32.497Z",
"duration": 1178,
"steps": [
{
"title": "browserContext.newPage",
"category": "pw:api",
"startTime": "2024-08-21T14:36:32.501Z",
"duration": 1174,
"steps": []
}
]
}
]
},
{
"title": "page.goto(https://sandbox.mabl.com/)",
"category": "pw:api",
"startTime": "2024-08-21T14:36:33.680Z",
"duration": 1231,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 5,
"column": 15
}
},
{
"title": "locator.getByRole('button', { name: 'new window' }).click",
"category": "pw:api",
"startTime": "2024-08-21T14:36:34.925Z",
"duration": 186,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 6,
"column": 57
}
},
{
"title": "After Hooks",
"category": "hook",
"startTime": "2024-08-21T14:36:35.112Z",
"duration": 24,
"steps": [
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:36:35.114Z",
"duration": 1,
"steps": []
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:36:35.116Z",
"duration": 1,
"steps": []
}
]
}
],
"errors": []
}
],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 4,
"column": 1
},
"parent": {
"title": "example.spec.js",
"_only": false,
"_requireFile": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 0,
"column": 0
},
"parent": {
"title": "",
"_only": false,
"_requireFile": "",
"parent": {
"title": "",
"_only": false,
"_requireFile": "",
"_use": [],
"_entries": [
null
],
"_hooks": [],
"_staticAnnotations": [],
"_tags": [],
"_modifiers": [],
"_parallelMode": "none",
"_type": "root"
},
"_use": [],
"_entries": [
null
],
"_hooks": [],
"_staticAnnotations": [],
"_tags": [],
"_modifiers": [],
"_parallelMode": "none",
"_fullProject": {
"project": {
"grep": {},
"grepInvert": null,
"outputDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\test-results",
"repeatEach": 1,
"retries": 0,
"metadata": {},
"name": "",
"testDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"snapshotDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"testIgnore": [],
"testMatch": "**/*.@(spec|test).?(c|m)[jt]s?(x)",
"timeout": 30000,
"use": {},
"dependencies": [],
"__projectId": ""
},
"fullConfig": {
"config": {
"configFile": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\playwright.config.js",
"rootDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"forbidOnly": false,
"fullyParallel": false,
"globalSetup": null,
"globalTeardown": null,
"globalTimeout": 0,
"grepInvert": null,
"maxFailures": 0,
"metadata": {
"actualWorkers": 1
},
"preserveOutput": "always",
"reporter": [
[
"C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\PlaywrightReporter.js",
null
]
],
"reportSlowTests": {
"max": 5,
"threshold": 15000
},
"quiet": false,
"projects": [
null
],
"shard": null,
"updateSnapshots": "missing",
"version": "1.46.1",
"workers": 4,
"webServer": null
},
"configDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"configCLIOverrides": {},
"webServers": [],
"plugins": [],
"projects": [
null
],
"cliArgs": [],
"cliListOnly": false,
"cliPassWithNoTests": false,
"cliFailOnFlakyTests": false,
"defineConfigWasUsed": true
},
"expect": {},
"respectGitIgnore": true,
"snapshotPathTemplate": "{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{-snapshotSuffix}{ext}",
"ignoreSnapshots": false,
"id": "",
"deps": []
},
"_type": "project"
},
"_use": [],
"_entries": [
null
],
"_hooks": [],
"_staticAnnotations": [],
"_tags": [],
"_modifiers": [],
"_parallelMode": "none",
"_fileId": "dd9b2edffe9b7501040c",
"_type": "file"
},
"type": "test",
"expectedStatus": "passed",
"timeout": 30000,
"annotations": [],
"retries": 0,
"repeatEachIndex": 0,
"_testType": {
"fixtures": [
{
"fixtures": {
"defaultBrowserType": [
"chromium",
{
"scope": "worker",
"option": true
}
],
"browserName": [
null,
{
"scope": "worker",
"option": true
}
],
"playwright": [
null,
{
"scope": "worker",
"box": true
}
],
"headless": [
null,
{
"scope": "worker",
"option": true
}
],
"channel": [
null,
{
"scope": "worker",
"option": true
}
],
"launchOptions": [
{},
{
"scope": "worker",
"option": true
}
],
"connectOptions": [
null,
{
"scope": "worker",
"option": true
}
],
"screenshot": [
"off",
{
"scope": "worker",
"option": true
}
],
"video": [
"off",
{
"scope": "worker",
"option": true
}
],
"trace": [
"off",
{
"scope": "worker",
"option": true
}
],
"_browserOptions": [
null,
{
"scope": "worker",
"auto": true,
"box": true
}
],
"browser": [
null,
{
"scope": "worker",
"timeout": 0
}
],
"acceptDownloads": [
null,
{
"option": true
}
],
"bypassCSP": [
null,
{
"option": true
}
],
"colorScheme": [
null,
{
"option": true
}
],
"deviceScaleFactor": [
null,
{
"option": true
}
],
"extraHTTPHeaders": [
null,
{
"option": true
}
],
"geolocation": [
null,
{
"option": true
}
],
"hasTouch": [
null,
{
"option": true
}
],
"httpCredentials": [
null,
{
"option": true
}
],
"ignoreHTTPSErrors": [
null,
{
"option": true
}
],
"isMobile": [
null,
{
"option": true
}
],
"javaScriptEnabled": [
null,
{
"option": true
}
],
"locale": [
null,
{
"option": true
}
],
"offline": [
null,
{
"option": true
}
],
"permissions": [
null,
{
"option": true
}
],
"proxy": [
null,
{
"option": true
}
],
"storageState": [
null,
{
"option": true
}
],
"clientCertificates": [
null,
{
"option": true
}
],
"timezoneId": [
null,
{
"option": true
}
],
"userAgent": [
null,
{
"option": true
}
],
"viewport": [
null,
{
"option": true
}
],
"actionTimeout": [
0,
{
"option": true
}
],
"testIdAttribute": [
"data-testid",
{
"option": true
}
],
"navigationTimeout": [
0,
{
"option": true
}
],
"baseURL": [
null,
{
"option": true
}
],
"serviceWorkers": [
null,
{
"option": true
}
],
"contextOptions": [
{},
{
"option": true
}
],
"_combinedContextOptions": [
null,
{
"box": true
}
],
"_setupContextOptions": [
null,
{
"auto": "all-hooks-included",
"title": "context configuration",
"box": true
}
],
"_setupArtifacts": [
null,
{
"auto": "all-hooks-included",
"title": "trace recording",
"box": true
}
],
"_contextFactory": [
null,
{
"scope": "test",
"title": "context",
"box": true
}
],
"_optionContextReuseMode": [
"none",
{
"scope": "worker",
"option": true
}
],
"_optionConnectOptions": [
null,
{
"scope": "worker",
"option": true
}
],
"_reuseContext": [
null,
{
"scope": "worker",
"title": "context",
"box": true
}
]
},
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\node_modules\\playwright\\lib\\index.js",
"line": 884,
"column": 39
}
}
]
},
"id": "dd9b2edffe9b7501040c-afd96d08a88ecfcf2eec",
"_poolDigest": "37500f500367e2c2037c76158069d8d577577210",
"_workerHash": "-37500f500367e2c2037c76158069d8d577577210-0",
"_projectId": "",
"_staticAnnotations": [],
"_tags": []
},
"step": {
"retry": 0,
"parallelIndex": 0,
"workerIndex": 0,
"duration": 5545,
"startTime": "2024-08-21T14:38:11.661Z",
"stdout": [],
"stderr": [],
"attachments": [],
"status": "passed",
"steps": [
{
"title": "Before Hooks",
"category": "hook",
"startTime": "2024-08-21T14:38:11.668Z",
"duration": 1587,
"steps": [
{
"title": "fixture: browser",
"category": "fixture",
"startTime": "2024-08-21T14:38:11.685Z",
"duration": 331,
"steps": [
{
"title": "browserType.launch",
"category": "pw:api",
"startTime": "2024-08-21T14:38:11.694Z",
"duration": 322,
"steps": []
}
]
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:38:12.018Z",
"duration": 32,
"steps": [
{
"title": "browser.newContext",
"category": "pw:api",
"startTime": "2024-08-21T14:38:12.023Z",
"duration": 23,
"steps": []
}
]
},
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:38:12.051Z",
"duration": 1204,
"steps": [
{
"title": "browserContext.newPage",
"category": "pw:api",
"startTime": "2024-08-21T14:38:12.056Z",
"duration": 1198,
"steps": []
}
]
}
]
},
{
"title": "page.goto(https://sandbox.mabl.com/)",
"category": "pw:api",
"startTime": "2024-08-21T14:38:13.260Z",
"duration": 4140,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 5,
"column": 15
}
},
{
"title": "locator.getByRole('button', { name: 'new window' }).click",
"category": "pw:api",
"startTime": "2024-08-21T14:38:17.409Z",
"duration": 123,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 6,
"column": 57
}
},
{
"title": "After Hooks",
"category": "hook",
"startTime": "2024-08-21T14:38:17.534Z",
"duration": 22,
"steps": [
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:38:17.535Z",
"duration": 1,
"steps": []
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:38:17.537Z",
"duration": 0,
"steps": []
}
]
}
],
"errors": []
}
} |
onStepEnd(test, step, result): Logs the end of a test step, including the result of the step.
| Code Block | ||
|---|---|---|
| ||
{
"test":{
"title": "basic test",
"_only": false,
"_requireFile": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"results": [
{
"retry": 0,
"parallelIndex": 0,
"workerIndex": 0,
"duration": 3094,
"startTime": "2024-08-21T14:33:03.207Z",
"stdout": [],
"stderr": [],
"attachments": [],
"status": "passed",
"steps": [
{
"title": "Before Hooks",
"category": "hook",
"startTime": "2024-08-21T14:33:03.218Z",
"duration": 2537,
"steps": [
{
"title": "fixture: browser",
"category": "fixture",
"startTime": "2024-08-21T14:33:03.237Z",
"duration": 933,
"steps": [
{
"title": "browserType.launch",
"category": "pw:api",
"startTime": "2024-08-21T14:33:03.245Z",
"duration": 925,
"steps": []
}
]
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:33:04.172Z",
"duration": 21,
"steps": [
{
"title": "browser.newContext",
"category": "pw:api",
"startTime": "2024-08-21T14:33:04.176Z",
"duration": 13,
"steps": []
}
]
},
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:33:04.194Z",
"duration": 1561,
"steps": [
{
"title": "browserContext.newPage",
"category": "pw:api",
"startTime": "2024-08-21T14:33:04.198Z",
"duration": 1557,
"steps": []
}
]
}
]
},
{
"title": "page.goto(https://sandbox.mabl.com/)",
"category": "pw:api",
"startTime": "2024-08-21T14:33:05.761Z",
"duration": 1291,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 5,
"column": 15
}
},
{
"title": "locator.getByRole('button', { name: 'new window' }).click",
"category": "pw:api",
"startTime": "2024-08-21T14:33:07.065Z",
"duration": 161,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 6,
"column": 57
}
},
{
"title": "After Hooks",
"category": "hook",
"startTime": "2024-08-21T14:33:07.228Z",
"duration": 30,
"steps": [
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:33:07.230Z",
"duration": 0,
"steps": []
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:33:07.231Z",
"duration": 0,
"steps": []
}
]
}
],
"errors": []
}
],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 4,
"column": 1
},
"parent": {
"title": "example.spec.js",
"_only": false,
"_requireFile": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 0,
"column": 0
},
"parent": {
"title": "",
"_only": false,
"_requireFile": "",
"parent": {
"title": "",
"_only": false,
"_requireFile": "",
"_use": [],
"_entries": [
null
],
"_hooks": [],
"_staticAnnotations": [],
"_tags": [],
"_modifiers": [],
"_parallelMode": "none",
"_type": "root"
},
"_use": [],
"_entries": [
null
],
"_hooks": [],
"_staticAnnotations": [],
"_tags": [],
"_modifiers": [],
"_parallelMode": "none",
"_fullProject": {
"project": {
"grep": {},
"grepInvert": null,
"outputDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\test-results",
"repeatEach": 1,
"retries": 0,
"metadata": {},
"name": "",
"testDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"snapshotDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"testIgnore": [],
"testMatch": "**/*.@(spec|test).?(c|m)[jt]s?(x)",
"timeout": 30000,
"use": {},
"dependencies": [],
"__projectId": ""
},
"fullConfig": {
"config": {
"configFile": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\playwright.config.js",
"rootDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"forbidOnly": false,
"fullyParallel": false,
"globalSetup": null,
"globalTeardown": null,
"globalTimeout": 0,
"grepInvert": null,
"maxFailures": 0,
"metadata": {
"actualWorkers": 1
},
"preserveOutput": "always",
"reporter": [
[
"C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\PlaywrightReporter.js",
null
]
],
"reportSlowTests": {
"max": 5,
"threshold": 15000
},
"quiet": false,
"projects": [
null
],
"shard": null,
"updateSnapshots": "missing",
"version": "1.46.1",
"workers": 4,
"webServer": null
},
"configDir": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index",
"configCLIOverrides": {},
"webServers": [],
"plugins": [],
"projects": [
null
],
"cliArgs": [],
"cliListOnly": false,
"cliPassWithNoTests": false,
"cliFailOnFlakyTests": false,
"defineConfigWasUsed": true
},
"expect": {},
"respectGitIgnore": true,
"snapshotPathTemplate": "{snapshotDir}/{testFileDir}/{testFileName}-snapshots/{arg}{-projectName}{-snapshotSuffix}{ext}",
"ignoreSnapshots": false,
"id": "",
"deps": []
},
"_type": "project"
},
"_use": [],
"_entries": [
null
],
"_hooks": [],
"_staticAnnotations": [],
"_tags": [],
"_modifiers": [],
"_parallelMode": "none",
"_fileId": "dd9b2edffe9b7501040c",
"_type": "file"
},
"type": "test",
"expectedStatus": "passed",
"timeout": 30000,
"annotations": [],
"retries": 0,
"repeatEachIndex": 0,
"_testType": {
"fixtures": [
{
"fixtures": {
"defaultBrowserType": [
"chromium",
{
"scope": "worker",
"option": true
}
],
"browserName": [
null,
{
"scope": "worker",
"option": true
}
],
"playwright": [
null,
{
"scope": "worker",
"box": true
}
],
"headless": [
null,
{
"scope": "worker",
"option": true
}
],
"channel": [
null,
{
"scope": "worker",
"option": true
}
],
"launchOptions": [
{},
{
"scope": "worker",
"option": true
}
],
"connectOptions": [
null,
{
"scope": "worker",
"option": true
}
],
"screenshot": [
"off",
{
"scope": "worker",
"option": true
}
],
"video": [
"off",
{
"scope": "worker",
"option": true
}
],
"trace": [
"off",
{
"scope": "worker",
"option": true
}
],
"_browserOptions": [
null,
{
"scope": "worker",
"auto": true,
"box": true
}
],
"browser": [
null,
{
"scope": "worker",
"timeout": 0
}
],
"acceptDownloads": [
null,
{
"option": true
}
],
"bypassCSP": [
null,
{
"option": true
}
],
"colorScheme": [
null,
{
"option": true
}
],
"deviceScaleFactor": [
null,
{
"option": true
}
],
"extraHTTPHeaders": [
null,
{
"option": true
}
],
"geolocation": [
null,
{
"option": true
}
],
"hasTouch": [
null,
{
"option": true
}
],
"httpCredentials": [
null,
{
"option": true
}
],
"ignoreHTTPSErrors": [
null,
{
"option": true
}
],
"isMobile": [
null,
{
"option": true
}
],
"javaScriptEnabled": [
null,
{
"option": true
}
],
"locale": [
null,
{
"option": true
}
],
"offline": [
null,
{
"option": true
}
],
"permissions": [
null,
{
"option": true
}
],
"proxy": [
null,
{
"option": true
}
],
"storageState": [
null,
{
"option": true
}
],
"clientCertificates": [
null,
{
"option": true
}
],
"timezoneId": [
null,
{
"option": true
}
],
"userAgent": [
null,
{
"option": true
}
],
"viewport": [
null,
{
"option": true
}
],
"actionTimeout": [
0,
{
"option": true
}
],
"testIdAttribute": [
"data-testid",
{
"option": true
}
],
"navigationTimeout": [
0,
{
"option": true
}
],
"baseURL": [
null,
{
"option": true
}
],
"serviceWorkers": [
null,
{
"option": true
}
],
"contextOptions": [
{},
{
"option": true
}
],
"_combinedContextOptions": [
null,
{
"box": true
}
],
"_setupContextOptions": [
null,
{
"auto": "all-hooks-included",
"title": "context configuration",
"box": true
}
],
"_setupArtifacts": [
null,
{
"auto": "all-hooks-included",
"title": "trace recording",
"box": true
}
],
"_contextFactory": [
null,
{
"scope": "test",
"title": "context",
"box": true
}
],
"_optionContextReuseMode": [
"none",
{
"scope": "worker",
"option": true
}
],
"_optionConnectOptions": [
null,
{
"scope": "worker",
"option": true
}
],
"_reuseContext": [
null,
{
"scope": "worker",
"title": "context",
"box": true
}
]
},
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\node_modules\\playwright\\lib\\index.js",
"line": 884,
"column": 39
}
}
]
},
"id": "dd9b2edffe9b7501040c-afd96d08a88ecfcf2eec",
"_poolDigest": "37500f500367e2c2037c76158069d8d577577210",
"_workerHash": "-37500f500367e2c2037c76158069d8d577577210-0",
"_projectId": "",
"_staticAnnotations": [],
"_tags": []
},
"step": {
"retry": 0,
"parallelIndex": 0,
"workerIndex": 0,
"duration": 2705,
"startTime": "2024-08-21T14:31:07.284Z",
"stdout": [],
"stderr": [],
"attachments": [],
"status": "passed",
"steps": [
{
"title": "Before Hooks",
"category": "hook",
"startTime": "2024-08-21T14:31:07.292Z",
"duration": 1483,
"steps": [
{
"title": "fixture: browser",
"category": "fixture",
"startTime": "2024-08-21T14:31:07.310Z",
"duration": 327,
"steps": [
{
"title": "browserType.launch",
"category": "pw:api",
"startTime": "2024-08-21T14:31:07.319Z",
"duration": 317,
"steps": []
}
]
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:31:07.639Z",
"duration": 18,
"steps": [
{
"title": "browser.newContext",
"category": "pw:api",
"startTime": "2024-08-21T14:31:07.643Z",
"duration": 11,
"steps": []
}
]
},
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:31:07.659Z",
"duration": 1116,
"steps": [
{
"title": "browserContext.newPage",
"category": "pw:api",
"startTime": "2024-08-21T14:31:07.663Z",
"duration": 1112,
"steps": []
}
]
}
]
},
{
"title": "page.goto(https://sandbox.mabl.com/)",
"category": "pw:api",
"startTime": "2024-08-21T14:31:08.781Z",
"duration": 1333,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 5,
"column": 15
}
},
{
"title": "locator.getByRole('button', { name: 'new window' }).click",
"category": "pw:api",
"startTime": "2024-08-21T14:31:10.127Z",
"duration": 177,
"steps": [],
"location": {
"file": "C:\\Users\\VinayChristopher\\OneDrive - Royal Cyber Inc\\Documents\\D drive\\Official\\TEAF\\playwrght\\index\\example.spec.js",
"line": 6,
"column": 57
}
},
{
"title": "After Hooks",
"category": "hook",
"startTime": "2024-08-21T14:31:10.306Z",
"duration": 33,
"steps": [
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:31:10.309Z",
"duration": 0,
"steps": []
},
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:31:10.311Z",
"duration": 0,
"steps": []
}
]
}
],
"errors": []
},
"result": {
"title": "browserType.launch",
"parent": {
"title": "fixture: browser",
"parent": {
"title": "Before Hooks",
"category": "hook",
"startTime": "2024-08-21T14:34:58.224Z",
"duration": 1530,
"steps": [
null,
{
"title": "fixture: context",
"category": "fixture",
"startTime": "2024-08-21T14:34:58.555Z",
"duration": 22,
"steps": [
{
"title": "browser.newContext",
"category": "pw:api",
"startTime": "2024-08-21T14:34:58.560Z",
"duration": 13,
"steps": []
}
]
},
{
"title": "fixture: page",
"category": "fixture",
"startTime": "2024-08-21T14:34:58.579Z",
"duration": 1175,
"steps": [
{
"title": "browserContext.newPage",
"category": "pw:api",
"startTime": "2024-08-21T14:34:58.584Z",
"duration": 1170,
"steps": []
}
]
}
]
},
"category": "fixture",
"startTime": "2024-08-21T14:34:58.241Z",
"duration": 312,
"steps": [
null
]
},
"category": "pw:api",
"startTime": "2024-08-21T14:34:58.247Z",
"duration": 305,
"steps": []
}
} |
There are primarily two ways to use the Custom Reporter API.
Define in playwright.config.js/ts
| Code Block | ||
|---|---|---|
| ||
import { defineConfig } from '@playwright/test';
export default defineConfig({
globalTeardown: './global-teardown',
}); |
Passing the parameter while executing the test command
| Code Block | ||
|---|---|---|
| ||
import {exec} from "child_process";
exec("npx playwright test --reporter="./myreporter/PlaywrightCustomReporter.js"") |
Writing Own Custom Reporter with custom execution of Playwright Script
Custom execution of Playwright script can also be done one step at a time using async handler of express by passing one step at a time in a loop. This allows to use a completely independent custom reporter. Below is an example:
CustomReporter.js
| Code Block |
|---|
class CustomReporter {
constructor() {
this.results = [];
}
logStep(stepIndex, command, status, duration, errorMessage = null) {
const stepResult = {
stepNumber: stepIndex + 1,
stepCommand: command,
status,
duration: `${duration}ms`,
timestamp: new Date().toISOString(),
errorMessage: errorMessage,
};
this.results.push(stepResult);
console.log(`[Step ${stepIndex + 1}] ${status.toUpperCase()}: ${command} (Duration: ${duration}ms)`);
}
saveReport() {
// Save or send the report as needed
console.log("Final Test Report:", JSON.stringify(this.results, null, 2));
}
}
export default CustomReporter;
|
playwrightCustomExecution.js
| Code Block |
|---|
import asyncHandler from "express-async-handler";
import {chromium} from "playwright";
import CustomReporter from "./customReporter.js";
// Custom Reporter
// @route POST /api/pw/exe
// @access Public
const execution = asyncHandler(async (req, res) => {
const {playwright_test_steps} = req.body;
if (playwright_test_steps) {
const reporter = new CustomReporter(); // Initialize reporter
let browser;
try {
(async () => {
browser = await chromium.launch({
headless: false,
});
const context = await browser.newContext();
const page = await context.newPage();
for (let i = 0; i < playwright_test_steps.length; i++) {
const command = playwright_test_steps[i];
const startTime = Date.now(); // Start time for execution
try {
await eval(command.replace("page", "page").replace("await ", ""));
const duration = Date.now() - startTime; // Calculate duration
reporter.logStep(i, command, "passed", duration);
} catch (error) {
const duration = Date.now() - startTime; // Calculate duration
reporter.logStep(i, command, "failed", duration, error.message);
await page.screenshot({path: `error-${i}.png`});
reporter.saveReport();
throw new Error(`Error executing test step ${i}: ${error.message}`);
}
}
await context.close();
await browser.close();
// Save the final report
reporter.saveReport();
res.json({
message: "Test steps executed successfully",
report: reporter.results,
});
})();
} catch (error) {
console.log("Error executing test steps", error.message);
if (browser) await browser.close();
// res.status(500).json({
// message: "Error executing test steps",
// error: error.message,
// report: reporter.results,
// });
}
} else {
res.status(401);
throw new Error("Invalid Script");
}
});
export {execution};
|
testResults.json
...
| language | json |
|---|
...
very versatile with report generation. It can be categorized in three ways:
In-built Reports and Third-Party Support:
Playwright offers built-in report generation features, including HTML and JSON reports, which provide insights into test results and performance. It also supports third-party reporters like Allure and Jest, which can be integrated to customize the reporting format and functionality.
Custom Report API of Playwright:
Playwright provides a Custom Report API that allows you to create tailored reports by hooking into test events. This API lets you capture detailed information about test execution, steps, and results, enabling you to design reports that meet specific needs or organizational standards.
For maximum flexibility, you can develop a completely custom reporting solution by directly interfacing with Playwright's test results and events. This approach involves writing your own code to collect, format, and display test data according to your unique requirements, giving you full control over the reporting process.