TestNG-XSLT, a small project of mine, has finally made it to 1.0. It’s currently tested only with Firefox 3.x and Internet Explorer 7, but it will probably work on most modern browsers. For the moment I thought it is more important to target the popular browsers and leave the more exotic ones for later.
Some of noteworthy features that I’ve added since the project start:
- Maven plugin with access to all transformer features and settings.
- SVG pie chart with the overview of the pass/fail/skip status.
- Configurable CSS style sheet – you can build your own “skin” by overwriting the built-in styles and setting the
testNgXslt.cssFile parameter. More about this here.
- Ability to change the report title, to generate information about test durations and other settings configurable through XSL transformer parameters.
- Other smaller or bigger issues and feature requests reported by users.
I want to thank all the guys that used this and came up with bugs, feature requests and suggestions: Andreas Guther, Xuan Ngo, Francis Henninger, Csongor Gyuricza, Andreas Kuhtz and others.
Last, but not least, I would like to mention that Haw-Bin Chai has managed to successfully integrate TestNG-XSLT in Lightest, a lightweight testing framework based on TestNG and Groovy.
The download is available here. If you use Maven, make sure you update your pom.xml to get the 1.0.9 version of the plugin.
I finally got the time to finalize the Maven plugin for TestNG XSLT. A detailed usage scenario is described here.
A few important notes:
- Integration with the Maven site (navigation link in the “Project Reports” section)
- CSS customization is performed by passing a
cssFile configuration parameter to the plugin
As mentioned in a previous post, starting with the 5.6 version of TestNG, we have the ability to get an XML report (in the testng-results.xml file) that takes in consideration most of the TestNG features that are not available in the JUnit-compatible XML results. Although my spare time was fairly limited lately, I finally managed to create a project for an XSL transformer to handle these reports, in order to provide a useful and user-friendly output for TestNG, as an alternative to the classic HTML reports.
The project is currently hosted on Google Code at http://code.google.com/p/testng-xslt and the current version is 0.2. For the moment you can only get it from the SVN repository, but that will change as soon as I’ll have some time to write some decent documentation and pack it with a download zip. Some sample generated reports are available at http://cosminaru.ro/test/testng-xslt.
To get started, you can call ant test in the project base directory to see it running with a sample set of data. Your output should be generated in the test/single/output subdirectory. You can also use the build.xml file as a sample of integrating this with Ant and the output of a TestNG test run. Please take in consideration that the module is relying on lots of XSL 2.0 features. As far as I can tell, the only current implementation for that is Saxon, but you don’t have to worry about that since is bundled in the lib directory of the project.
Yes, the 5.6 version of TestNG is finally out and contains several new features and a couple of bug fixes for both the core and IDE plugins. The complete list of these can be found here.
One of the noteworthy fresh features of 5.6 that I would like to talk about is the new XML reporter. This reporter provides additional TestNG-specific info that is not available in the JUnit-compatible report. It aims at helping automatic results reporting systems by promoting a consistent XML format, while also offering the foundation for obtaining user-friendly results.
The documentation for this feature along with a sample XML output is available here. You might have noticed that the XML is class-centric. This means, that no matter how you specify your test scenarios, you will always get the output built around the classes whose methods were the subject of unit testing.
As you might have expected, this reporter is included in the list of default listeners so you don’t have to enable it by hand. It’s output will be written in the same directory as the rest of the default reports (the TestNG output folder) in a file named testng-results.xml.
There are however a couple of settings and tweaks that can be used to fine-tune the reporter to best fit your needs, as you might have noticed in the docs. These properties are passed in a JavaBeans style as we will see later.
One of these properties that you might find useful is the fileFragmentationLevel integer. This property provides a way to control the structure of the generated XML files by allowing you to separate the files into smaller pieces if you consider that one single XML file is too big to chew at once.
For the beginning, if you have more than one suite in your tests, you can choose to have the results for each suite in a separate XML file (by setting fileFragmentationLevel to 2). This way, the main file will only reference the suite files, as you can see here:
<testng-results>
<suite url="Suite1/testng-results.xml"/>
</testng-results>
The suite file will only contain the <suite> element with the same structure as before:
<suite name="Suite1">
<groups>
...
</groups>
<test name="test1">
<class name="com.test.TestOne">
...
</class>
</test>
</suite>
If you want to go even deeper with this, you can have your suite test-cases written in separate files, for which you will have to set fileFragmentationLevel to 3.
In this case, besides the previous effects, you will also get the test-cases files referenced from the suite files:
<suite name="Suite1">
<groups>
...
</groups>
<test url="test1.xml"/>
</suite>
and as you probably guessed, test1.xml looks like this:
<test name="test1">
<class name="com.test.TestOne">
...
</class>
</test>
The next question would be “how the hell do I set fileFragmentationLevel and the other properties?”. The docs also mentions this stuff and you should read that before doing anything else. However, I can detail a little bit about the most common ways to configure this reporter.
The first thing you need to know is that in 5.6, along with the other changes, there is also a new method to inject custom report listeners providing fine-grained access to their behavior. The foundation of this is the new -reporter argument that can be passed from the command line. The docs about this are here. This will inject a reporter like -listener would, but it also allows you to set some properties on it.
In our case, if for example we would like to change fileFragmentationLevel and outputDirectory of the XML reporter, we would pass to the command line something like this:
-reporter org.testng.reporters.XMLReporter:fileFragmentationLevel=2,outputDirectory=/usr/temp/results
There is also an Ant equivalent for this type of configuration, and it can be achieved by using the inner <reporter> element, which in our case would be something like this:
<target name="test" depends="build">
<testng ...>
<reporter classname="org.testng.reporters.XMLReporter">
<property name="fileFragmentationLevel" value="2"/>
<property name="outputDirectory" value="/usr/temp/results"/>
</reporter>
...
</testng>
</target>
This kind of configuration can be used starting with TestNG 5.6 for any of your custom listeners, not just the XML reporter. There are however a couple of things here that must be considered. First of all, the types that are currently supported as reporter properties are limited to the following set: java.lang.String, int, boolean, byte, char, double, float, long, short. Besides, as you might have noticed, it might be a little tricky to pass strings containing commas as parameters, although it can be easily worked-around.
Another thing, strictly related to the XML reporter, is that the built-in reporter can only use the default settings, an issue also mentioned by the documentation. You can however inject a new instance of this reporter manually with one of the methods described above (yes, the above code will create a new instance, and it will not configure the default one). However, in order not to get into conflicts with the built-in reporter you have two options: one is to disable the default listeners and the other one is to change the output directory of the manually injected instance (they both worked for me).
Hope you find this useful.