Entries from April 2007 ↓

Yes, the 5.6 version of TestNG is finally out and contains several new features and a couple of bug fixes for both the core and IDE plugins. The complete list of these can be found here.
One of the noteworthy fresh features of 5.6 that I would like to talk about is the new XML reporter. This reporter provides additional TestNG-specific info that is not available in the JUnit-compatible report. It aims at helping automatic results reporting systems by promoting a consistent XML format, while also offering the foundation for obtaining user-friendly results.
The documentation for this feature along with a sample XML output is available here. You might have noticed that the XML is class-centric. This means, that no matter how you specify your test scenarios, you will always get the output built around the classes whose methods were the subject of unit testing.
As you might have expected, this reporter is included in the list of default listeners so you don’t have to enable it by hand. It’s output will be written in the same directory as the rest of the default reports (the TestNG output folder) in a file named testng-results.xml.
There are however a couple of settings and tweaks that can be used to fine-tune the reporter to best fit your needs, as you might have noticed in the docs. These properties are passed in a JavaBeans style as we will see later.
One of these properties that you might find useful is the fileFragmentationLevel integer. This property provides a way to control the structure of the generated XML files by allowing you to separate the files into smaller pieces if you consider that one single XML file is too big to chew at once.
For the beginning, if you have more than one suite in your tests, you can choose to have the results for each suite in a separate XML file (by setting fileFragmentationLevel to 2). This way, the main file will only reference the suite files, as you can see here:

<testng-results>
  <suite url="Suite1/testng-results.xml"/>
</testng-results>

The suite file will only contain the <suite> element with the same structure as before:

<suite name="Suite1">
  <groups>
  ...
  </groups>

  <test name="test1">
    <class name="com.test.TestOne">
    ...
    </class>
  </test>
</suite>

If you want to go even deeper with this, you can have your suite test-cases written in separate files, for which you will have to set fileFragmentationLevel to 3.
In this case, besides the previous effects, you will also get the test-cases files referenced from the suite files:

<suite name="Suite1">
  <groups>
  ...
  </groups>
  <test url="test1.xml"/>
</suite>

and as you probably guessed, test1.xml looks like this:

<test name="test1">
  <class name="com.test.TestOne">
  ...
  </class>
</test>

The next question would be “how the hell do I set fileFragmentationLevel and the other properties?”. The docs also mentions this stuff and you should read that before doing anything else. However, I can detail a little bit about the most common ways to configure this reporter.
The first thing you need to know is that in 5.6, along with the other changes, there is also a new method to inject custom report listeners providing fine-grained access to their behavior. The foundation of this is the new -reporter argument that can be passed from the command line. The docs about this are here. This will inject a reporter like -listener would, but it also allows you to set some properties on it.
In our case, if for example we would like to change fileFragmentationLevel and outputDirectory of the XML reporter, we would pass to the command line something like this:

-reporter org.testng.reporters.XMLReporter:fileFragmentationLevel=2,outputDirectory=/usr/temp/results

There is also an Ant equivalent for this type of configuration, and it can be achieved by using the inner <reporter> element, which in our case would be something like this:

<target name="test" depends="build">
  <testng ...>
    <reporter classname="org.testng.reporters.XMLReporter">
      <property name="fileFragmentationLevel" value="2"/>
      <property name="outputDirectory" value="/usr/temp/results"/>
    </reporter>
    ...
  </testng>
</target>

This kind of configuration can be used starting with TestNG 5.6 for any of your custom listeners, not just the XML reporter. There are however a couple of things here that must be considered. First of all, the types that are currently supported as reporter properties are limited to the following set: java.lang.String, int, boolean, byte, char, double, float, long, short. Besides, as you might have noticed, it might be a little tricky to pass strings containing commas as parameters, although it can be easily worked-around.
Another thing, strictly related to the XML reporter, is that the built-in reporter can only use the default settings, an issue also mentioned by the documentation. You can however inject a new instance of this reporter manually with one of the methods described above (yes, the above code will create a new instance, and it will not configure the default one). However, in order not to get into conflicts with the built-in reporter you have two options: one is to disable the default listeners and the other one is to change the output directory of the manually injected instance (they both worked for me).

Hope you find this useful.