logo

Test Driven Development FAQ

Feedback is welcome, as always. rf.eerf@sartnap.
What questions are missing here ?

Generalities

What is TDD ?

TDD stands for Test-Driven Development. Despite the name, it's more about software design than about testing. It does not replace any other test phase, such as integration, functional or system test. It just allows developers to deliver "clean code that works".
TDD relies on automated Unit Tests that allow executing the code in a controlled environment.

TDD is the cornerstone of eXtreme Programming. However, it is independent from that method and can be used with any other development process, agile or not. Notice that eXtreme Programming now names them Programmer Tests instead of Unit Tests.

The TDD cycle

TDD is an iterative and incremental development technique ; it implies writing the test code before the production code, one test at a time, verifying that the test fails before writing the code that will make it pass.

red - green - refactor. This is the TDD mantra.

The red and green steps are very important as they allow testing the test: failing the test first and then having a green is an evidence that it's actually testing something.

TDD terminology

Unit
The term unit is voluntary vague. It's the thing (function, method, class, closure, set of functions ...) being tested.
Testcase
Function or method that tests one behavior of the code
Testsuite
Set of Testcases, typically grouped by fixture.
Testcase
Function or method that tests one behavior of the code
Testsuite
Set of Testcases, typically grouped by fixture.
Fixture
Execution Environment for one or more testcases. Fixtures are usually generated via a setUp() method and destroyed in tearDown().
Assertion
Function or macro allowing to compare the expected value against the actual one.
setUp and tearDown
setUp() is a method of the TestCase class in jUnit that can be redefined in your Test class and used to create the execution environment for all test cases in the class. TearDown() is the opposite of setUp(). It is called to cleanup the test environment.
setUp() and tearDown() are called before and after every test case.
Refactoring
Refactoring is a source code manipulation to improving the design of the code, without altering its external behavior.
Unit
What is being tested, ranging from a single method to a set of classes.
Mock object
An object implementing the same interface as, or subclassing a class of the production code, and exhibiting a deterministic behavior. Mock objects can be used to replace objects that rely on expensive resources (network, DB, disks ... )
Red bar and green bar
This comes from the graphical version of JUnit, where progression bar shows the execution of the tests, and goes from green to red when a test fails .
Test-first
Test-First-Programming is another name for Test Driven Development. It could be used when the design has been defined first, and there is almost no refactoring.

xUnit Frameworks

TDD typically involves using what's called an Unit Test Framework. It just enables easily writing and executing unit tests.

Every programming language has its xUnit version, even Unix shells, see ShUnit and shUnit2.

Ron Jeffries maintains a list of frameworks on XProgramming.com, and there is another one on Wikipedia.

Questions

These are the Frequently Asked Questions on the TDD mailing list.

What is the structure of a test ?

Structue your tests with the triple 'A': Arrange, Act, Assert.
Arrange: establish the fixture for the test class
Act: invoke the function (method) you're testing
Assert: assert to verify the behavior of the code is correct

You can also assert before to act to ensure the final assertion was effectively due to the action.

How many assertions per testcase ?

Some recommends to have one assertion per test to make the tests simpler.
"One Assert per Test" is simply a proxy for a principle we can probably all agree on: "Each test should test one thing".

To mock or not to mock ?

If you code bottom-up, then you should be able to avoid mock objects. Nevertheless those can be handwritten, or generated by a tool in the build chain, see, for example, easyMock.org.
Refer to mockObjects.com for an introduction and a list of implementations in various languages.
Refer this article by Martin Fowler, mocks aren't stubs, for a discussion differenciating between mocks and stubs.
See also Mock Roles, not objects.

Interaction testing vs. state testing

Those are two styles of TDD, interaction testing heavily relies on mock objects while state testing is based on ... object states.

I'm extracting a new class, shall I create new tests for it ?

This question was debated several time on the TDD list. You do not have to write new tests for the extracted class, but you can...
The consensus is that you can safely extract a class without writing new tests since it is already tested thru the initial class. However if you extracted the class to reuse it via another client, then you might want to add new tests targetting the extracted class for your new client class may use it another way.

How do I test private methods ?

There are several schools here:

Where do I start ?

Write a test first ;o)

Where can I find a good example of TDD ?

The bowling game demonstrates emergent design (in Java); it's also available in Haskell, C# ...

Top-down or bottom-up ?

TDDing bottom-up is creating building blocks while top-down is more of a divide-and-conquer strategy.
Top-down gives control while bottom-up gives abstraction.

Should I write tests for my getters and setters ?

If you're applying TDD stricto senso, never adding any line of code without a failing test, then all your accessors are covered by tests. Moreover, you're not testing your language, so you do not have to test them. Finally, test them if this is haunting your nights.

Doesn't TDD break OOP rules, for e.g. encapsulation?

When you feel the need to make things public only for testing, the code is telling you to sprout out a new class. The fields or methods that you want to expose will fit better into a separate class, of which they are the primary responsibility; the current class can delegate, then, those responsibilities to a private instance of the new class.

Patterns (and anti-patterns)

Kent Beck's book (see below) about TDD introduces several patterns related to Unit testing:

A long list of TDD anti-patterns and set up a wiki.

TDD in a non-agile environment

Even if your company process mandates you to detail the design in a document that needs to be reviewed and approved before you can start coding, you can take advantage of TDD in the form of Test First Programming. That is once you have your design and are allowed to start coding, do it test first.

TDD and databases

How to test-drive code that access a database ?
There are two possibilities: first you can mock the classes accessing the DB, or you can use transations to rollback your requests at teardown time to isolate tyour tests.

TDD and legacy code

Michael Feathers defines legacy code as "code without tests".
Refer to his book "Working Effectively with Legacy Code" for tips and techniques on how to to deal with such code.

TDD vs. BDD

BDD could be nextgen TDD, or "TDD done well".

Should the tests be refactored ?

Yes, tests are just code.

How do I test an abstract class ?

If you applied TDD rigourously, it is already covered by your tests.
Otherwise, you can use an abstract test case.

Rules of thumb

A test is not a unit test if:

  1. It talks to the database
  2. It communicates across the network
  3. It touches the file system
  4. It can't run correctly at the same time as any of your other unit tests
  5. You have to do special things to your environment (such as editing config files) to run it.
Tests that do such things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes).

On the negative side

On cost

TDD is hard to learn and takes a lifetime to master. It produces up to twice as much code as production code. However, unit tests code is much simpler.
But the automatic test suite it creates is a real safety net when it comes to refactoring.

Coplien on TDD

Jim Coplien is saying that TDD leads to architectural disaster in the long run and debated with Martin Fowler at JAOO07...

References

Links

Books

Disclaimer: All the opinions expressed here are either mine or those published in the TDD mailing list, which are subject to possible misunderstanding. Thanks a million to Kent Beck, Ron Jeffries, Carl Manaster, Sven Gorts, James Carr, Bill Wake, Charlie Poole, Martin Fowler, Michael Feathers and all those that are not cited here for they thoughts on TDD that allowed the creation of this FAQ.


rf.eerf@sartnap:otliam -
v0.19 - may. 2oo8