Another aspect you can look at is coverage trends, especially if you are trying to build up test coverage on a legacy codebase. The principle is, as you develop or change code, the overall test coverage (at an overall, component, and package level) should never go down. If it doesn’t then you are probably not adding enough coverage for the new code you are writing.

I’ll also assert that I think its more important to do a review of unit tests than of code, especially if you have good unit test coverage and good metrics tools (static analysis, dependency analysis, etc) output.

Reviews would focus on:
* testing the proper scenarios
* finding missing scenarios
* performing adequate asserts

If one was thinking agile and lean, you could move the low-level designs to being unit test javadoc, so your unit test reviews would be equivalent to low level design reviews. This would eliminate a form of documentation and put the info in a place where it is easier kept up to date (if it gets out of date – theoretically the testcase will fail).

Your point about unit testing simply not getting done later in the cycle is a good one, but by making a conscious shift in project planning, and by having a good way of measuring the tests, I wonder if we could make it work.

I used the term “white box” testing as an alternative to “unit tests”, in an attempt to separate the two slightly. In an ideal world it might be possible to thoroughly unit test (and get the benefits you highlighted), and still have some time to focus on rounding out the code coverage later on with more white box tests. The trick is then deciding how we measure whether we’ve done enough in that first round of unit tests.

I also agree that additional metrics such as code complexity are useful. I haven’t seen xRadar before and will certainly take a look.

There are several reasons:
* Provides immediate feedback to code changes. A full unit test suite for a large product should take about 10 minutes. individual test methods should run around a tenth of a second.
* Forces you to write maintainable code (you have to write thing in small chuck and think through dependencies and abstraction in order to write an effective unit test)
* Guards against regression. The specific logic paths that your code should take are continuously run every time a change to the code is made. This lets people other than the authors make changes with confidence.
* Documents your low level design (you should write and name your tests to described the desired behavior of your code. Google behavior driven development (BDD) for more on this.

Also, you can’t wait until function tests are done to apply unit tests. In my experience, if you wait until after the code is developed (and especially tested) to add unit tests, then it just simply won’t get done. There is no need and no pressure to add them. the next work item will always take precedence. You have to do unit tests while and even before you code.

You are right that relying coverage tools alone can be misleading. That’s why you have to use a variety of data points as quality indicators, such as code complexity, static analysis, peer reviews, and dependency analysis. A good tool for aggregating this data is xRadar (http://xradar.sourceforge.net/).

You are also right that unit test alone is not good enough to guarantee quality. Higher level component tests (testing larger units to ensure interactions between classes), acceptance/functional testing (validating overall flows work and are meeting the business requirements) are all vital pieces of the puzzle (as well as system test, and all the other varieties).