Jake Herringbone

A test-driven modern language

Here’s another in my recent series of posts about ways I think the Java language could support modern coding practices.

An obvious practice that we use extensively today is unit testing and test-driven development, and again, Java and other languages don’t provide built-in support. Instead, we have a few libraries to create test code, execute it, and provide mock dependencies, and then some standards for naming and directory layout that help us organize the test code and correlate it with production code.

What sorts of facilities would our dream language have for testing?

In Java, tests have a lot of repetitive code, because they are just classes. All test methods are in the same form, namely that they are non-static, take no arguments, marked as a test via an annotation or naming convention, throw all checked exceptions, and return void. We should probably have a test keyword that starts a block, and this should act like a test.

Just like the keywords extends and implements, we should have a way to associate two objects with a new relationship: tests. This way, we can easily see for a given class, what classes test it, and IDE’s can consistently navigate between a class and its test(s). It also would allow some conveniences. Instead of having to mark methods package-private so they are visible to the unit test, we could make private methods visible to tests, like an implicit C++ “friend” relationship.

So, with our two new keywords we have an example like this:

class Foo {
  private int helperMethod() {
    //stuff
  }
}
class FooTest tests Foo {
  test helper {
    int i = new Foo().helperMethod();
    assertThat(i, equals(10));
  }
}

Now we have created a test class and some tests. Notice the assertion in that test – where do the assertThat() and equals() come from? Instead of importing some utility classes, maybe the tests keyword could also cause my FooTest class to extend from a base Test class rather than from Object by default, so these could be implemented there. Or if the language had mixins, the test methods could come from a mixin and not require polymorphism (we don’t care that our test class may be cast to a Test type).

The example also uses a Hamcrest-style expectation, to make the assertion more fluent.

The same thing could also be done if tests could be written directly in the class they test, and the compiler will have to ignore it when not in testing mode. In that case, the mixin would be automatically added to classes that contain tests, so that the assertions are available:

class Foo {
  // implicit mixin Asserts
  int calculateDay(Date date) {
    ...
  }
  test calculateDay {
    // using the enclosing instance, requires that it have a default constructor
    int day = calculateDay(new Date()); 
    assertThat(day, equals(31));
 
    // probably more realistic
    int otherDay = new Foo().calculateDay(new Date());
    assertThat(day, equals(31));
  }
}

What more can we do to make writing tests the most convenient and fastest way to code? For one, we could have a built-in test runner. Having marked our tests with a new syntax, we can find all the tests and execute them using an equivalent program to the compiler. In that test running mode, we can relax the security model of the runtime, avoiding some common problems with testing sensitive code.

Once we have our test runner, we can lower the bar to getting good testing infrastructure in a project. One major pain in the butt is instrumenting the code to get the line-level test coverage. Because libraries like JUnit execute code that’s been compiled by the normal compiler, we have to do something funny like twiddle the bytecode after compilation, or use a custom classloader to do it on the fly. If the language understood testing, then it could also always provide coverage data when the tests are executed. It would be easier for IDE’s to show how much of your code is executed in the tests, as well.

Finally, there is the ever-annoying issue of setting up dependencies. The language needs access to different libraries at test-time, and we also want to enforce that production code may not have dependencies on test code. With language-level support for testing, there can be compile-time checks that test code is not used from production code, and we can have a second path of libraries passed only to the test runner.

Does this seem like a good idea, or does the language step over the line here and take over the job of a framework? Should we encourage testing private methods this way, or does it violate encapsulation principles?

Tags: nil, testing