Backstage Java (BSJ) - Tutorial

This page contains a very brief tutorial in the use of BSJ. It makes use of the distribution of the BSJ compiler found on the homepage (or here). The commands used in this tutorial will assume a current working directory of the root of the compiler distribution.

To run the BSJ compiler, you simply need to invoke the class edu.jhu.cs.bsj.compiler.impl.tool.bsjc.BsjC as described in the readme.txt file in the distribution. If you are on a system that supports sh, you should be able to use the provided bsjc script.

Hello, world!

To introduce BSJ metaprogramming, we will begin with a simple Hello World application. Begin by entering the following code into a file helloworld/HelloWorld.bsj.

package helloworld;
public class HelloWorld {
    public static void main(String[] arg) {
        [:
            BlockStatementListNode list = context.getAnchor().<BlockStatementListNode>getNearestAncestorOfType(
                    BlockStatementListNode.class);
            list.addFirst(<:System.out.println("Hello, world!");:>);
        :]
    }
}

This code can then be compiled into a simple Hello World Java application by use of the BSJ compiler:

$ ./bsjc -d bin helloworld/HelloWorld.bsj

Please be aware of the fact that the BSJ compiler is, at this point, not tuned for performance; it may take several seconds to compile your program.

Once compilation is finished, a file named HelloWorld.class will have been produced. Furthermore, a directory named bsjgensrc will be present which displays the Java source which was generated by the BSJ compiler. At this time, the BSJ compiler operates by executing the metaprograms in the source file and producing .java output which is then compiled by a normal Java compiler. This permits the user to inspect the code which was generated to determine if it looks correct.

The example code above contains a metaprogram delimited by [: and :]. This code is executed by the BSJ compiler in order to modify the source and produce the corresponding .java file. The variable context is bound in scope to an object which permits the metaprogrammer access to various resources. In the case of this metaprogram (and most others), the context variable is used to obtain the anchor of the metaprogram: a node in the AST representing that metaprogram’s position. The metaprogram then obtains the nearest ancestor of type BlockStatementListNode (which is the list of statements in the main method) and inserts a System.out.println statement.

The code inserted into the BlockStatementListNode takes the form of a code literal (similar to the quasiquote from LISP macros). A code literal is a piece of object program code delimited by the <: and :> operators. The code literal is designed to make syntax construction easier; for instance, the code

<: 5 :>

is equivalent to the code

context.getFactory().makeIntegerLiteralNode(5)

assuming that context is in scope. (The actual code transformation is somewhat more complicated and prevents the metaprogrammer from needing to pass context into each and every method.)

Of course, BSJ metaprograms can perform considerably more sophisticated operations than merely inserting literal code. The following example will demonstrate how methods can be generated from the presence of fields.

Two Classpaths

Metaprograms in BSJ are compiled over two classpaths: the metaprogram classpath and the object program classpath. The object program classpath is the classpath with which Java users will be familiar: it defines those libraries with which compiled code will be linked and thus will be available at runtime. The metaprogram classpath identifies the libraries which are available to the metaprograms which are executed by the compiler.

We will now create a library that our metaprogram can use. Create a file metautils/Utils.bsj with the following contents.

package metautils;

import java.util.*;
import edu.jhu.cs.bsj.compiler.ast.*;
import edu.jhu.cs.bsj.compiler.ast.node.*;
import edu.jhu.cs.bsj.compiler.ast.node.list.*;
import edu.jhu.cs.bsj.compiler.ast.node.meta.*;

public class Utils {
    public static void createMethodsAndField(ClassMemberListNode members, BsjNodeFactory factory,
            String methodSuffix, String fieldName) {
        // First, add an increment method for each field
        List<ClassMemberNode> newMembers = new ArrayList<ClassMemberNode>();
        for (ClassMemberNode member : members) {
            if (member instanceof FieldDeclarationNode) {
                FieldDeclarationNode fieldDecl = (FieldDeclarationNode)member;
                for (VariableDeclaratorNode decl : fieldDecl.getDeclarators()) {
                    String name = decl.getIdentifier().getIdentifier();
                    IdentifierNode methodIdent = factory.makeIdentifierNode(name + methodSuffix);
                    IdentifierNode varIdent = factory.makeIdentifierNode(name);
                    newMembers.add(<:
                        public void ~:methodIdent:~() {
                            ~:varIdent:~ += 1;
                        }
                        :>);
                }
            }
        }
        members.addAll(newMembers);
        // Now add a field
        members.add(<:private int ~:factory.makeIdentifierNode(fieldName):~;:>);
    }
}

The portions of the code delimited by the ~: and :~ operators are splices; they are similar to antiquotes in the LISP macro system. The expression contained within is evaluated in the scope of the surrounding metaprogram code and is expected to evaluate to some Node type. In the above cases, the expressions evaluate to identifier nodes; those identifier nodes are then used in generating the AST when the code literal is compiled.

The createMethodsAndField method is a contrived library function which will add an increment method for each field declared in the provided class member list. It will then add a field to the class (for which no increment function will be generated). To use this library function, we must compile it and provide it in the metaprogram classpath of a later compilation. Execute the following (or its equivalent) to compile the utilities class:

$ ./bsjc -d metabin metautils/Utils.bsj

Next, put the following in a file named example/Example.bsj:

package example;

#import metautils.Utils;

public class Example {
    private int x;
    [:
        Utils.createMethodsAndField(
                context.getAnchor().<ClassMemberListNode>getNearestAncestorOfType(ClassMemberListNode.class),
                context.getFactory(), "Inc", "y");
    :]
}

Finally, compile this file ensuring that the utility is available on the metaprogram classpath.

$ ./bsjc -d bin -mcp metabin example/Example.bsj

Upon successful compilation, you should be able to find the Example.java sources in the bsjgensrc directory. It contains two fields – x and y – and a method xInc.

Metaprogram Dependencies and Difference-Based Metaprograms

Consider the following code:

package example;

#import metautils.Utils;

public class Example2 {
    private int x;
    [:
        Utils.createMethodsAndField(
                context.getAnchor().<ClassMemberListNode>getNearestAncestorOfType(ClassMemberListNode.class),
                context.getFactory(), "Inc", "y");
    :]
    [:
        Utils.createMethodsAndField(
                context.getAnchor().<ClassMemberListNode>getNearestAncestorOfType(ClassMemberListNode.class),
                context.getFactory(), "Up", "z");
    :]
}

A question arises about the semantics of the above code: which metaprogram runs first? If the top metaprogram runs first, then the variable y is present when the second metaprogram is running and a yUp method is produced. Conversely, running the second metaprogram first would produce a zInc method. So which result occurs?

The answer is, in fact, neither. Difference-based metaprogramming does not view these metaprograms as transformation functions that must be composed; they are viewed as difference generators that produce differences which need to be merged. Upon compiling the above code, neither yUp nor zInc will be generated. This is because both metaprograms are executed against copies of the original AST (the one with only the x field); the changes that they make are then merged into another copy.

If, on the other hand, one of these results is desired, it can be accomplished by use of a preamble declaration in each metaprogram. If, for instance, we wish the second metaprogram to run over the output of the first metaprogram, we can write the following:

package example;

#import metautils.Utils;

public class Example2 {
    private int x;
    [:
        #target foo;
        Utils.createMethodsAndField(
                context.getAnchor().<ClassMemberListNode>getNearestAncestorOfType(ClassMemberListNode.class),
                context.getFactory(), "Inc", "y");
    :]
    [:
        #depends foo;
        Utils.createMethodsAndField(
                context.getAnchor().<ClassMemberListNode>getNearestAncestorOfType(ClassMemberListNode.class),
                context.getFactory(), "Up", "z");
    :]
}

In the above code, the first metaprogram is a member of the target foo. A target is simply a set of metaprograms; the metaprogram is in the set if it includes the corresponding target declaration. Any metaprogram may be a member of multiple targets at once. The second metaprogram indicates that it depends on that target; this means that it is executed over the output of all of the metaprograms in that target. In this way, the second metaprogram will see the y variable and thus generate the yUp method.

The names of targets are qualified either by the fully-qualified name of the class that contains them (if they are in a class) or the package name and compilation unit name of the file that contains them (if they are not in a class). Declarations of #target must be simple names, but declarations of #depends are permitted to be either simple or fully qualified.

A cycle in the metaprogram dependency graph is, of course, an error.

Meta-annotations

BSJ also supports a more declarative metaprogramming style in the form of meta-annotations. Meta-annotations are similar to Java annotations except in that (1) they use a different syntax, (2) they are only present at compilation and are stripped before .class files are generated, and (3) they may imply the existence of a metaprogram.

For example, consider the following code:

package example;

#import edu.jhu.cs.bsj.stdlib.metaannotations.*;

public class Point {
    @@Property private int x;
    @@Property private int y;
}

In the above code, @@Property indicates that a given variable should have a public getter and a public setter. Compiling this code produces a Java class with those methods included. The code which generates the getter and the setter is found in the class edu.jhu.cs.bsj.stdlib.metaannotations.Property, which must be present on the metaprogram classpath at compile time (which it is, as it is part of the BSJ standard libraries included with the compiler distribution). This class is an implementation of the BsjMetaprogramMetaAnnotation interface specified in the BSJ API and thus can be used as a meta-annotation in the compilation of the Point class above.

A further benefit of meta-annotation-driven metaprograms is that they are capable of abstracting over targets and dependencies. For instance, the definition of the Property class indicates that any use of it is a member of the target property; in the case above, the fully-qualified name of this dependency for the two @@Property instances is example.Point.property. This is particularly convenient in the following rendition of the Point class.

package example;

#import edu.jhu.cs.bsj.stdlib.metaannotations.*;

@@GenerateConstructorFromProperties
public class Point {
    @@Property private int x;
    @@Property private int y;
}

The GenerateConstructorFromProperties meta-annotation code will create a constructor which takes one argument for each getter (not each field) on the associated class. The declaration of this meta-annotation class indicates that it depends on the property target of the class in which it is positioned; this allows it to ensure that the getters generated by the @@Property meta-annotations are present when it generates its changes. And So On…

The BSJ compiler is still in a fragile state; type errors in metaprograms, for instance, often produce stack traces rather than helpful error messages. For a number of examples of usage, however, you may wish to consult the unit tests in the git repository (discussed on the home page). You may also wish to investigate the API Javadocs to get a feel for the metaprogramming environment that BSJ provides.