Unit Testing

The changes I currently make to Babel-17 for the jump from v0.21 to v0.3 reach deeply into the current code base. This makes me think about how to verify that the changes are consistent with my spec of Babel-17. Obviously, an important corner stone for interpreter / compiler correctness is a test suite. Most of the tests in this test suite can just be treated like unit tests for ordinary Babel-17 programs. This leads naturally to the question, how should unit testing work in Babel-17 ?

I am a strong advocate of providing programming language support for unit testing. Especially for Babel-17 this seems obvious: Babel-17 is dynamically typed, and although you can now (in v0.3) have modules, data encapsulation and abstract datatypes, there is only minimal static type checking in Babel-17 that merely ensures that the types you talk about really exist. You will still need unit testing to see that your types behave like you expect them to. Writing unit tests will be a standard task for every serious programmer who uses Babel-17, and therefore Babel-17 should provide language support for it. If you are saying, hey, this argument is bullshit, because for example version control is also a routine task, but best be left to external tools, then you might be right; but maybe you are very wrong 🙂

An important feature of testing is that the testing code does not affect the original production code. Many people interpret this as an argument against language support for unit testing, but actually only language support can guarantee the separation of production and testing code.

So, Babel-17 v0.3 will provide the following language level support for unittesting:

  • a new keyword unittest
  • you can define modules that contain unittest as part of their path, like in
    module com.obua.util.orderedset.unittest

    This module would typically test functionality of the module com.obua.util.orderedset. If you’d like to distinguish further the tests of this module, you can name your modules like this:

    module com.obua.util.orderedset.unittest.functionality1
    module com.obua.util.orderedset.unittest.functionality2
    module com.obua.util.orderedset.unittest.functionality3.sub1
    module com.obua.util.orderedset.unittest.functionality3.sub2

    and so on. Any module that has unittest in its path is called a unit test module. Note that a module path cannot repeat parts of itself; in particular, only a single component of the path can be unittest

  • you can use the unittest keyword as part of a module name also for the
    import statement, but only if it is issued from a unit test module
  • the last statement of a (non-unittest) module can be a unittest statement, like in
    module com.obua.util.orderedset

    This has the same effect of defining a module com.obua.util.orderedset.unittest, but with the difference that it shares the namespace of its surrounding module. In particular it can see the private definitions and type definitions (and the inner values of those types) of its surrounding module.

  • The pragma #assert is only executed and checked when running unittests, never in production code.
  • The new pragma #catch takes a constructor name (like InvalidArgument) and an expression. It signals an error if the evaluation of the expression does not yield an exception with that name. It also is only executed when running unittests.

Taken together this allows for the flexible creation of unit tests, while at the same time completely shielding production code from testing code.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: