The Art of Unit Testing and TDD in Java

Roy Osherove separates good unit testing from test-driven development, which he traces to extreme programming and Kent Beck's book Test-Driven Development by Example and calls a personal practice. He defines a unit as a unit of work that starts at a public API and ends in a return value, a noticeable state change or a third-party call, the only place for mock objects. He argues for one test per end result and questions claims that unit tests make code easier to debug, maintain and understand, as when 50 tests stop compiling after a constructor change.

Roy Osherove contrasts manual test rigs with xUnit frameworks and introduces JUnit 4 with a test of a string calculator's add method laid out as arrange, act, assert. He explains his three-part test names (unit of work, scenario, expected behavior) and why the before, after, before class and after class attributes usually point to integration tests. He defines unit tests as running in memory under full control and keeps them in a separate package from integration tests, giving a safe green zone of tests that should always pass.

In IntelliJ, Roy Osherove shows string calculator tests that use helper methods called make calc and assert adding to avoid repetition and keep the tests maintainable. He then compares ways to test that a line throws an exception in JUnit: a try/catch block, the expected parameter of the test annotation, which he advises against because any line can throw, and an expected exception rule. With the rule, only the lines after the expect call are supposed to throw the exception.

Roy Osherove gives the three rules of test-driven development: start with a failing test, make it work in the simplest way possible, then make it better. He adds a subtract method to the string calculator test-first, choosing simple values such as one and two and deciding the order of the parameters while the code does not exist yet. He makes the test fail by returning minus one before implementing the method, and shows the IntelliJ run configuration that runs all JUnit tests in the package.

Roy Osherove judges how simple a solution is by how close it is to the beginning of the method and how little it is indented. He test-drives a parse method on the string calculator, starting with an empty string that returns zero, and treats each hard-coded answer as an assumption that the next failing test, such as a single number, proves wrong. He connects these small steps to incremental development in micro iterations, with new working code that could be released every day.

Roy Osherove answers why tests are written before the code: to test the test, to think about the problem before the solution, and because there is never time to write them after the code. He says TDD makes developers try to prove that their code doesn't work, that the tests remove the fear of refactoring, and that doing the simplest thing avoids writing code nobody uses. He also explains why he has only one failing test at a time and writes the planned tests as comments.

Roy Osherove describes an experiment at a big company where two teams adapted the same features of a billing system, one with TDD and one without. The TDD team took 14 days instead of seven to implement the features but released in 24 days instead of 26, partly because its unit tests showed where integration bugs were not, and it had 90% fewer bugs in production. He says many companies don't try this because they fear investing without seeing the return for months.

Roy Osherove explains how to test a call to a third party with a string calculator that must send each sum to a slow logger, by adding a logger interface and a handwritten fake logger class. He defines a fake as something that looks like something else, a mock as a fake you assert against and a stub as any other fake, and allows at most one mock per test. The fake is passed in through constructor injection, and a test name such as add always calls logger shows that a mock object is involved.

Roy Osherove writes the add always calls logger test first, as if everything already existed, with a fake logger that saves the text in a public written field and is passed to the string calculator's constructor. Because the other tests create the calculator through the make calc helper, the new constructor argument changes one place, where a fake logger is passed in as a stub. He sees only the new test fail, makes add call the logger, and has to add throws Throwable to the tests, which he dislikes.

Roy Osherove contrasts his simple terminology with the book xUnit Test Patterns, which gives many different names to things that look like other things. He only uses fakes: a fake that is asserted on is a mock, any other fake is a stub, and terms such as test spy can be learned later.

Roy Osherove adds a requirement that a web service is called when the slow logger throws an exception, for example when the disk is full on a machine. The fake logger becomes a stub with a will throw method that sets the exception to simulate, and a fake web service becomes the mock object that the test asserts on. He writes the add when logger throws calls web service test with an out of memory error, and says the test name tells you what to do when the fakes get confusing.

Roy Osherove explains that isolation frameworks, which he prefers not to call mocking frameworks, automate handwritten fakes: they create fakes, save call arguments for verification and simulate exceptions or return values. He lists EasyMock, jMock, PowerMock, JMockit and Mockito, and uses jMock's expectations and EasyMock's record and replay to show how strict fakes, which fail on unexpected calls, break tests for the wrong reason. He recommends EasyMock's nice mocks and shows Mockito, which is non-strict by default, with verify, doThrow and parameter matchers such as any string.

Roy Osherove shows Mockito's argument captor for checking that a trace message with specific attributes was sent to the logger, by capturing the argument in a verify call and asserting on its value. He calls this clumsy and compares it with a handwritten fake class, which has to change whenever a method is added to the logger interface. A participant suggests implementing equals on the trace message, which he says can be the most readable way when the object is simple.

Roy Osherove separates constrained isolation frameworks, which cannot override static or private methods, from unconstrained ones such as JMockit and PowerMock, which can fake them. Without them, he applies extract and override to a string calculator that calls a static logger: the call moves into a protected call logger method, which a testable subclass in the test overrides to save the text in a public field. He also explains seams, keeping logic out of the extracted method, and starting legacy code with integration tests before refactoring.

Roy Osherove explains why tests should not depend on the current date and time, and why extract and override or a time provider interface in every constructor does not suit a cross-cutting concern like time. Instead he uses a static time provider class in the production code that tests can force to a given time and reset afterwards, for example in an after method. For the machine name in the next exercise he suggests extract and override.

Roy Osherove discusses an exercise with a login manager that creates a new slow logger, where the test has to prove that the logger's write method is called. He compares extracting a protected method that returns the slow logger, which needs a fake subclass of it, with extracting the whole call into a protected method, which he usually prefers as simpler.

Roy Osherove shows the static logger test written with PowerMock, Mockito and JUnit, using the run with annotation with the PowerMock runner, an annotation that loads the static logger class differently, and verify static before each verified call. He then shows the same tests in JMockit, which has to be loaded before JUnit and uses non-strict expectations and verifications blocks. He does not usually recommend JMockit because calls are repeated twice and tests get wordy, and suggests starting the exercises with PowerMock.

Roy Osherove shows how to make the write method of a static logger throw an exception with PowerMock: after mock static, doThrow with an out of memory exception is set on the static logger class, and the next call to the static logger is the one that throws. He notes that this API is not obvious at first and that a matcher such as any string tells the reader more.

After the exercises Roy Osherove asks how unconstrained frameworks felt, concludes that they were much more complicated than refactoring, and explains that an API for faking private and static methods, constructors and other legacy code features has to be as complex as legacy code. He says tools like PowerMock and JMockit let you test code without changing it but give up the readability and maintainability that refactoring brings, so they should be used like a nuclear weapon.

Drawing on his book in progress Beautiful Build, Roy Osherove calls releases the best form of compilation and separates build controllers such as TeamCity or Jenkins, build agents and build scripts by what each should know. He recommends starting with a shipping skeleton, a hello world application that goes through the whole release chain, and covers cumulative builds sharing artifacts, snapshot dependencies, build script injection and parameterized FinalBuilder scripts. He also discusses the bus factor, build waves that let developers resume coding once CI passes, and why breaking the build should not be discouraged.

Roy Osherove ends the course by pointing to artofunittesting.com for more videos and free material, to his Twitter account RoyOsherove and to osherove.com for courses. He mentions that he is writing books about beautiful builds and about team leaders in software.