Some thoughts on testing Acceleo Java generators

I’ve tried the last few weeks to found the best way to write automated tests for my Acceleo generators and have come to some thoughts and findings that may be interesting.

The first resource I found on the subject was this Tumblr post from Stephane Begaudeau about how unit testing the Acceleo templates and queries. Even though Stephane work was close to an achievement, there was still work to be done on Acceleo API.

Also I realized that my needs were much closer to integration testing : follow a “black box” approach and have the ability to automatically launch non regression tests on a blueprint model when my generators change. Thanks to my previous findings on launching Acceleo from Maven, I would have the base infrastructure to do this.

All the code samples of this post and more details can be found on GitHub into the https://github.com/lbroudoux/acceleo-maven-sample/ repository, into the com.github.lbroudoux.acceleo.uml.java.jpa-tests folder.

3 levels of integration testing

So the use-case I’ll try to cover is the following : I have Acceleo generators that produce a bunch of Java and Xml files from a model and I’m going to write test to check that produced files are correct. I quickly realize that this checkings my be done at 3 different levels.

Byte code level

The first level is the byte code one and is available for the Java part of my use-case (you can of course extend this to any compiling language). This first level is interesting because it can be quickly achieved, just use the compiler and the reflection APIs provided by the JDK to check : that methods are generated with correct params, that fields are presents and so on …

However, this method has some limitations :

  • the code you produce has to compile ! This may seems odd but in large projects you may have many generators producing only parts of the whole puzzle and it may be difficult to make every generated unit compiling without pulling a lot of dependencies. Also sometimes you may want your generated code not to compile in order to force the developer to write something clever 😉
  • compilation is a destructive process ! Some elements found in the sources dissapear when transformed into byte code … How to : retrieve parameter names, check javadoc presence or ensure that annotations are all there ?

Syntax tree level

The second level answers these limitations by providing a representation of the tree of directives found in the source files. This is generally called an “Abstract Syntax Tree” that may be produced and visited using APIs. The most famous one is the DOM APIs that represent the AST of an Xml document but analogous tools exists for Java and compiled language (we’ll see 2 of them in next section)

Using this AST allows us to check details that dissapear after the compilation step. It also allows us to verify that some statements (assignations, loops, switch, …) are done in respect of the coding rules/conventions. Sadly enough, an AST is “Abstract” that means that some details are still missing depending on the performance of the available tooling. Sometimes, it is necessary to go deeper to the third level…

String level

This last level is the obvious but tedious one : work at the string level (because after all everything we produce is text !). The kind of tools employed is more primitive : pattern and regexp matchers, string comparison and file readers. Every language offers its shortcuts for that (personally, I love Groovy grep, file and =~ operator a lot ;-))

Possible solutions for Java

The Syntax tree level is maybe the most ignored one, so I’m going to detail it a bit more … After a quick Google search, 2 solutions seems available.

Javaparser

Javaparser is a project hosted on http://code.google.com/p/javaparser/ and that seems quite inactive for a time. However, it does most of the job right.

The usage principle is the following : you just have to parse an input stream in order to obtain a compilation unit AST.

CompilationUnit stuffCu = null;
FileInputStream in = new FileInputStream("../Stuff.java");
try{
   // Parse the file.
   stuffCu = JavaParser.parse(in);
}
finally{
   in.close();
}

Then, in order to traverse/browse the produced compilation unit, you’ll have to write a visitor implementation like this :

public class ClassSummaryVisitor extends VoidVisitorAdapter<Object>{
   // [..]
   @Override
   public void visit(ClassOrInterfaceDeclaration n, Object arg){
      summary = new ClassSummary(n);
      super.visit(n, arg);
   }
}

You finally have to call your visitor and, for example, later retrieve the produced representation :

// Use a visitor to build a summary.
ClassSummaryVisitor csVisitor = new ClassSummaryVisitor();
csVisitor.visit(stuffCu, null);
cs = csVisitor.getSummary();

Eclipse JDT

JDT stands for Java Devloper Tools and its the module responsible of all the Java related things in the Eclipse platform. It is used by project management module, editors, code style checker, etc in the IDE.

The main class in JDT that has to be used for our purpose is ASTVisitor. It’s usage is quite analogous to Javaparser except the initialization part that is a little bit trickier :

// Creates an input stream for the file to be parsed.
File javaFile = new File("../Stuff.java");
BufferedReader in = new BufferedReader(new FileReader(javaFile));
final StringBuffer buffer = new StringBuffer();
String line = null;
while (null != (line = in.readLine())) {
   buffer.append(line).append("\n");
}
      
// Parse and get the compilation unit.
ASTParser parser = ASTParser.newParser(AST.JLS3);
parser.setKind(ASTParser.K_COMPILATION_UNIT);
parser.setSource(buffer.toString().toCharArray());
parser.setResolveBindings(false);
CompilationUnit stuffCu = (CompilationUnit)parser.createAST(null);

For the rest of the usage, I recommend you checking the GitHub project.

Maven to the rescue

We’ve got our base tooling and thanks to Maven and JUnit we can automate all of this.

Test process

First, be sure to place the parsing code above into a JUnit test case class initialization block and then write some test methods with assertions like below :

@Test
public void testAnnotationPresence(){
   // Check that JPA annotation is present.
   assertTrue(cs.hasAnnotation("javax.persistence.Entity"));
}

@Test
public void testFieldsPresence(){
  // Check that two fields have been produced.
  assertTrue(cs.hasField("foo"));
  assertEquals(ModifierSet.PUBLIC, cs.getField("foo").getModifiers());
  assertTrue(cs.hasField("bar"));
  assertEquals(ModifierSet.PUBLIC, cs.getField("bar").getModifiers());
}
   
@Test
public void testFieldsJavadoc(){
  assertTrue(cs.getField("foo").getJavadoc().toString().contains("Description of the property foo."));
}

Then, build yourself a blueprint model including all the generation cases you may allow and follow my previous posts in order to automatically enable the generation from this model using Maven.

Be sure that Maven launches the Acceleo generation before the execution of your JUnit test. For that, I recommend you to bind the generation onto the generate-test-resources phase of your build lifecycle.

Sample project

I mentionned above the sample project I’ve committed to GitHub. In this project, you will found :

  • A blueprint Uml model testModel.uml at project root that uses the generators from my previous posts,
  • A Maven pom.xml files that generates files into /target/test-resources directory
  • A Junit test case using Javaparser into /src/test/java/com/github/lbroudoux/acceleo/uml/java/jpa/files
  • A Junit test case using Eclipse JTD into /src/test/java/com/github/lbroudoux/acceleo/uml/java/jpa/files
  • Samples visitors for Javaparser and JDT into /src/main/java/com/github/lbroudoux/japa|jdt

Conclusion

Javaparser and Eclipse JDT are great tools for going into details of the source code and allow the checking of many things that dissapear with compilation. However, I have found limitations on both in the support of line and blocks comments that are not Javadocs :

  • JDT fully ignore them and give only information on the start and end line of blocks (useless !),
  • Javaparser tries to handle them but a bug into its parser make him lose the starting point of line comments if not followed by a Java instruction (a ‘;’ character is enough)

This feature would have been usefull for checking – for example – that my Acceleo templates were actually providing some protected area for code to be inserted !

The whole tool chain (AST + JUnit + Maven + Acceleo) makes the non regression checks on generators a breeze mainly if you plug them into a Continous Integration Server (such as Jenkins) in order to have thet checks trigerred by a modification on your Maven module containing your generators !

Let me know if it help some of you … Do not hesitate sending me feedback or other ideas on generators test automation !

Advertisements

One thought on “Some thoughts on testing Acceleo Java generators

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s