Time for some education again. This time it’s Test Driven Development with Venkat Subramaniam (@venkat_s)

Disclaimer: This is my own notes and my interpretation of things, NOT the teacher’s. I recommend you to take this course if you have the opportunity.

Table of Contents

Influences of TDD on Agility and Sustainability

It’s difficult to be Agile if we don’t do feedback driven development. If your objective is to build what your customer wanted. Without feedback from the customer, you will fail, and you still need to build what they still want.

Feedback is critical for you to succeed both when you are writing test and from your customer. When you show a customer what you have done, they will rarely jump up and down of joy, but instead say something like: “It looks nice, but can we change this and that”. This is called the observer effect. So the minute they see it they want to change it.

Observer effect – The minute they see it they want to change it.

Another thing you want to have is small deltas for each change. This is crucial for TDD, because if you take to large steps you will miss the final goal.

Circles of feedback

There are two circles of feedback, the circle of expectations and the circle of relevance. The circle of expectations states that code that worked yesterday should work today. The circle relevance asks if it’s relevant to the customer.

Development speed

TDD is not about high speed, but sustainable speed. The goal is to go equally fast at all times, and not only for the beginning of the project.

Cost of TDD

There are two costs of TDD. The cost of writing and the cost of learning.

Cost of the actual act of writing tests

The amount code produced is normally a ratio 3-5 times test code as production code. This is not actually a large time cost, developers are usually fast at writing on the keyboard. The time consuming part is to be able to come up with new test-cases that drives the code and design.

Cost of learning it

You must unlearn what you have learn, and relearn it the TDD way. This can be very hard to do, especially when you have work many years without doing TDD. So this is almost always a challenge.

One way to ease it is to use pair programming and actively remind each other to practice TDD.

Cost of not doing it

The problems you have today with a lower development pace as more features are added. This is to repeat old habits that you, in your heart, know don’t work.

Insanity – to repeat what does not work.

Types of tests

• Testing (manual)
• Verification (automated)

Testing are done manual to challenge the system and see that it works and feels right. This is important since we can’t write a unit test on the feeling of the application. Every thing else is  verification. Verification should be done automatically so automation is important in that sense.

We will focus on verification tests when doing TDD.

Automated testing is the software equivalent of exercising.
Most people will tell you it is good, but lacks the discipline to do it.

TDD vs Non TDD

Dr. Laurie Williams had an experiment with two teams, team 1 and team 2.

Team 1 works without TDD. Team 2 with TDD. The first features, both teams starts to work on it.
Team 1 finish the features first. Team 2 is 16 % slower to finish it.

Both teams then got new features to add to the previous ones. Now team 2 finish first. Team 1
finish it later, but they still got bugs in their code.

Local optimization vs. global optimization

Toyota changed from local optimization to global optimization. This gave them a lot better overall quality and process.

So look for global optimization instead of local.

When not to do TDD

If you are doing a prototype, don’t use TDD, learn from it, throw it away and reimplement the real solution with TDD.

If it should be maintainable use TDD. It saves time and money.

Test Driving your Design

By test driving your design you get the following:

1. Regression of your application
2. Design of your application

Don’t do JDD, Jesus Driven Development, the lord have better things to do then get your code to work. Use TDD instead and be sure that your code will do what you want.

If it is hard to get under test -> The design of the code sucks.
If the code is hard to test -> It usually means we have not understand the code.

Types of tests

Black box

Normally done by testers and business people that are viewing the system as a black box from outside. This is not the kind of tests that are done when you driving your development with tests.

White box

This is the kind of test you use when driving your development. White box testing is testing where you know the code that you’re tests are running against.

4 kinds of tests to write

• Positive – Happy path
• Negative – Unhappy path
• Exception – The right kind of exception
• Performance – If it’s important write a test for certain constants.

Write down test-cases on paper as we go along. This list will be growing and shrinking as time goes by. Strike out tests as they are implemented or discarded. Don’t write them down in your test class and ignore them using @Ignore!

What’s a unit test?

A unit test is a test on a unit of code.

A unit of code is the smallest piece of code that do useful work.

• Don’t test setter and getters.
• Don’t write tests for what a class have, write tests of what it does.
• Focus on the behaviour of the code.
• Don’t test the state, state is not self-serving, grow the state from the behaviour.

A unit test whispers how the code should be used.

Test first vs. unit testing

Write a test first, then minimum code to make it pass. If code is removed the test should not pass.

If you try to write code after the code, you will not have the time to write down all test-cases that describes the behaviour.

TDD is a skill, not a talent. So practise by doing it and retrospect on it. Do code reviews on the code AND the tests!

When using TDD, fake the implementation so it can grow the interface first.

Test first coding

If you can’t write a test for it, your test is to large or your code is to coupled.

If you don’t know how something works, play with it in a prototype directory. Learn and hack away.

Throw it away when you have learned from it. Write test for it when using it in your project the TDD way.

Benefits

• It grows the skin before the guts
• Evolves the code on what it should do
• The code becomes robust
• It verifies that the code works
• Confidence that it works
• Decouples the code from it’s neighbours
  • High cohesion and low coupling
• Form of documentation, by giving examples of how it’s used.

Good code is like a joke. It should be self-explained. Don’t explain it if people don’t understand, instead refactor it.

Examples of regression benefit

public int divide(int t, int n) {
    return t / n;
}

We test this code for an exception in the following way.

@Test
public void divideByZero() {
    try {
        math.divide(100, 0);
        fail("Division by Zero");
    } catch (Exception e) {
        assertTrue("Exception caught", true);
    }
}

If we change int to a double, the exception is no longer thrown. A double that is divided by zero simply returns INF. So this test is really useful to catch the wanted behaviour and alert us if someone changes that behaviour.

Why TDD?

So way should we use TDD? There are two main reasons for that:

1. Clean code that works
2. Predictability

Both of them are highly desirable when doing software development.

Program with intention

When you write code, the intention of what it will do should be captured in the design and code. A reader should understand what it intend to do just by looking at the code.

Two phase design

To help us get started we need still to do some design.

Strategic design

Strategic design is where you tries to identify different aspects and how they are dependent of each other. So just like a good old UML diagram. But the difference here is that you do just enough upfront design (EUFD) instead of a big up front design (BUFD).

Tactical design

Is where you cut out the low level details driven by TDD. This will change your strategic design as well and that is the reason to avoid BUFG and use EUFD instead.

Don’t do BUFD, it’s bad. Instead do EUFD.

Example

Build a Tic-Tac-Toe for 2 human player. What strategic design could we make and with parts do we identify? Write down test-cases in a task list. Follow a set of practices and principles to get on the right track.

Practices

• Start with a canary test (see goodbye hello world for an example of that)
• Don’t argue (with yourself or others) if a test is valid or not, just jot it down
• Take small steps
• Test behaviour, not state
• A good design will need fewer tests
• Think of positive, negative and exception tests
• Write minimum of code to make the test pass
  • Code is fast to make the test pass
  • Not writing untested code
  • Gives focus on the interface that we’re evolving
• Let the test fail first, then write minimum code to make the test pass
• What if the test passes first? Ensure that tests passes for the right reasons
• When write tests or writing code, you will think of more test conditions. Jot them down on your task list

Principles

• YAGNI – You Aren’t Gonna Need It
• Keep code and test DRY – Don’t Repeat Yourself
• Ensure the test are isolated from each other
• Test should be FAIR
  • Fast
  • Automated
  • Isolated
  • Repeatable
• Keep test DRY an isolated set-up can help
• Never refactor code when there is a red bar (failing tests)
• Either refactor before the change or after the change
  • Refactor should take us from green to green.
• Any time all test passes, it’s a great time for check in
• Use SLAP – Single level of abstraction principle

List of test tasks

Example of test task that we come up with. Tasks that is strike-through are implemented while tasks in italic are the one that we omitted.

• Place Peg
• Check if there is a winner
• Check game board
• Set first player
• Change player
• Move a peg
• Add player
• Place peg of occupied position
• Place peg out occupied
• Place peg of row range
• Place page out of row range
• Place page out of column range

Mocking out dependencies

First, try to knock-out mocks and avoid them if possible. Don’t mock out the predicable dependencies (helpers, utils, etc.). When the dependency is slow and unpredictable, you may consider to mock it. Use mocking when its expensive to use the real object. Also use mocks to mock out web-services, file systems, databases and low level OS function (if possible).

Mock objects are used to:

1. Simulate a behaviour
2. Simulate an ill behaviour

Types of mocks

1. Dummy – Returns values as close to null or 0 as possible
2. Stub – Returns what it was told to return
3. Spy – The real object, but can see and count interactions and also stub out methods
4. Mock – A smart stub that you can verify calls and call order on
5. Fake – A little behaviour simulator for a certain test

DIP – Dependency Inversion Principle

Depend on abstractions instead of concrete implementations. Gives good seams for mocking. This also gives a good and modular design for the system as whole.

Some things to think about when doing mocking

• Don’t build one mock to rule them all. Instead build specific mocks for the test, one mock per usage
• Don’t bloat the created interface if not necessary
• Don’t mock if the code is easy to work with, fast and predicable
• Mock only the code you directly depends on
• Use mocks to simulate ill behaviour
• Spy method out in the unit under test

Hands-on tips

• We don’t need to write tests for everything in the code, UI, setter/getter, etc.
• UI may need test for critical things.
• A ratio of 3-5 times more test code then production code.
• Testing the UI begin at the model Level->Controller->View
• The goal is not to write of lots of test, it’s to write meaningful test. Start with the behaviour/action/result.
• If you putting for loops in your test, you writing the test wrong. Stop and rethink.
• A test should have no more than one independent assert (Single assert rule). Don’t do Act->Assert->Act->Assert. Independent assert = one block of asserts.
• Memento pattern can be used to ease the testing by injecting the state you want to begin testing from.
• Don’t be ceremonial over Given When Then and similar practices and processes. Use the most self explaining and minimalistic style you can come up with. Especially with naming, use short describing names.

Spock frameworks

Spock is a testing framework written in groovy but can test Java code.

Example of spock tests:

def 'score for a vowel'() {
  expect:
    1 == scrabler.scoreLetter(vowel)
  where:
    vowel << ['a', 'e', 'i', 'o', 'u']
}

def 'score for words'() {
  expect:
    score == scrabler.score(FINNISH, word)
  where:
    word   | score
    'fish' |   7
    'in'   |   3
    'his'  |   5
}

def 'score for an incorrect spelled word'() {
  def spellChecker = Mock(SpellChecker)
  spellChecker.isCorrect('nis') >> false
  scrambler.setSpellChecker(spellChecker)

  expect:
    0 == scrabler.score(FINNISH, 'nis')
}

TDD with JavaScript for AngularJS

JavaScript is the easiest language to mock and test. Still it’s the language that is least tested.

What makes JavaScript hard to test?

• Coupling between modules, functions and the mess we make
• Us – We only assume that it is hard to test and hence we don’t write tests

The language is really easy to spy out. Just replace a function with custom one. An example using mocha.

describe('Example', function(){
  var stubWasCalled = false;
  var myObject = new Object();

  beforeEach(function(done) {
    myObject.myFunc = function() {
      stubWasCalled = true;
    }
  });

  describe('doStuff', function(){
    it('MyFunc should be called', function(){
      myObject.doStuff();
      stubWasCalled.should.equal(true);
    });
  });
});

Decoupling

AngularJS is designed with testing in mind. First do a prototype and the do it test driven.

Principles and practices

Angular is written with principles of testing. By using injection it’s easy to mock out requests and other dependencies.

Example

Use angular-mocks for mocking out requests and other angular types.

We are using mocha and chai as the testing framework as it’s not as verbose as jasmine. Karma will be used as the test-runner.

We are using nodejs to ease the development. So to set-up the environment lets do the following:

npm install -g karma-cli
npm install --save-dev karma karma-mocha karma-chai karma-phantomjs-launcher
npm install --save angular
npm install --save-dev angular-mocks
npm install karma-clear-screen-reporter --save-dev

Then set-up Karma by running karma init, take mocha (we will later add chai), choose no for Require.js, choose PhantomJS, default for rest.

Edit karma.conf.js to the following:

frameworks: ['mocha', 'chai'],
...
files: [
  'node_modules/angular/angular.js',
  'node_modules/angular-mocks/angular-mocks.js',
  './public/javascripts/*.js',
  './test/**/*.js'
],

Now we are good to go, so now we need to create our first test. So lets create the tasksContollerTest by doing the following:

mkdir test
touch test/tasksContollerTest.js

Let’s start with the canary test:

describe('TasksContollerTest', function() { 
  it('canary test passes', function() {
    expect(true).to.eql(false);
  });
});

Now lets run the canary test by issuing the following command, this will be running in the background and monitor the changes for automatic test running.

karma start --reporters clear-screen,dots

The test should fail, so now we got a running test environment. Correct the test and now we can start with the first test. The controller should have empty tasks on create.

We need to set-up the controller first, then test that we don’t have any tasks in the task list.

describe('TasksControllerTest', function() {
  var controller;
  
  beforeEach(module('todoApp'));
  
  beforeEach(inject(function() {
    controller = $controller('TasksController');
  }));

  it('canary test passes', function() {
    expect(true).to.eql(false);
  });

  it('controller has a tasks model which is empty on start', function() {
    expect(controller.tasks).to.eql([]);
  });
});

So onwards to the next test. The method getTasks should return tasks from the server.

We don’t want to deal with ajax call and server dependency right now. So we need some way to mock out this dependency. Fortunately angular provides a nice mock for this, the _$httpBackend_. So we simply needs to inject it instead of the regular $http service.

So we need to change the beforeEach to the following:

  beforeEach(inject(function($controller, _$httpBackend_) {
    controller = $controller('TasksController');
    $http = _$httpBackend_;
  }));

Then add the test

  it('getTasks should return the tasks from the server', function() {
    var sampleTasks = [
      {name: 'task2', year: '2015', month: 5, day: 20},
      {name: 'task1', year: '2015', month: 5, day: 20}
    ];
    
    $http.expectGET('/tasks')
         .respond(200, sampleTasks);
         
    controller.getTasks();
    $http.flush();
    expect(sampleTasks).to.include(controller.tasks[0]);
  });

We uses the expectGET to set the expectation and the return values. Because the call is asynchronous we need to call flush to get the callbacks trigged. Finally we asserts that we got the right respond.

The next test is that tasks returned should be in sorted order. In this case we need to implement a sorting function, by first writing a test for it. We need a test for each sorting order. I this case year, month, day and name. After that we need to make sure that getTasks is calling the sorting function, by writing another test for that. The last part is that we need to validate in some way that getTasks is called when the document is ready. We write a test for this that validates that the method registered for execution when the document is read is getTasks.

  describe('DocumentReadyTest', function() {
    var functionProvided;
    var controller;
    
    beforeEach(inject(function($controller, $document) {
      $document.ready = function(fun) {
        functionProvided = fun;
      }
      controller = $controller('TasksController');
    }));
    
    it('document ready called with getTasks', function() {
      expect(functionProvided).to.eql(controller.getTasks);
    });
    
  });

The final controller looks like this:

var TasksController = function($http, $filter, $document) {
  var controller = this;
  
  var sortTasks = function(tasks) {
    var orderBy = $filter('orderBy');
    return orderBy(tasks, ['year', 'month', 'day', 'name']);
  }
  
  var updateTasks = function(data) {
    controller.tasks = controller.sortTasks(data);
  }
  
  var getTasks = function() {
    $http.get('/tasks')
         .success(updateTasks);
  }
  
  controller.getTasks = getTasks;
  controller.sortTasks = sortTasks;
  controller.tasks = [];
  
  $document.ready(getTasks);
}

angular.module('todoApp', [])
       .controller('TasksController', TasksController);

One thing to notice here is that we don’t uses the $scope and that we are declaring each function in the own var and setting them to the controller later. We uses the same technique for the controller itself. By breaking it up in small pieces we are also separating the concerns.

Functional testing

Functional testing is often done in a black box manner. The testers tests the system from the given specifications and examples. Much of the functional testing can be automated and later serve as a regression suite and documentation over the system.

Functional testing vs. unit testing

The main difference is the scope and the form of testing. The table below describes the major differences.

	Functional	Unit-testing
Granularity	Coarse	Fine
Driver	Testers	Programmers
Language	Fluent	Code-like
Running speed	Varies	Fast
Focus	Relevance	Expectation

Functional testing is on a higher level

Focusing the testing on the expectation of the code or the behaviour of the application from the users point of view. Instead of the small units that build the application.

The code should meet the customer expectation, not the programmer. Functional testing should answer the question Are we building the right thing. Instead of unit testing that answers the question Are we building the thing right.

BDD vs. TDD

BDD making the code meet the customer expectations. This is done by focusing on the behaviour.

Remember, it’s not the syntax that is important, it’s the intent.

Fit and FitNesse

Fit Framework for integration testing developed by Ward Cunningham. It uses a tabular form to describe the input and expected output. FitNesse is a wiki implementation around Fit created by Uncle Bob.

If tabular form is not available, you can use easyb, RSpec, Cucumber or SpecFlow. These tools uses a more natural language.

An example of that test form:

Given ‘Customer has an active account’
when ‘customer deposits $100’
then ‘balance will goes up with $100’

Functional tests can be executable documentation!

UI Automation

If you need to do UI level testing try to keep it to a minimum.

For smoke tests only, all other tests should already be done on the controller level or below.

Continuous integration

Include the CI process in your TDD cycle to be the following:

1. Write a test
2. Make it pass
3. Check it in for CI
4. Refactor

The most common excuse that people don’t check in frequently is that the think the integration may require some manual actions. If you flip the situation over and check in the code as often as it is possible, the deltas is so small and that makes the integration go smooth instead.

Integration issue, if it hurts do it often.

If you do it often it’s no big deal.

1. Start to compile the code
2. Run all unit tests, if it fails the build fails.
3. Run functional test, integration test and code coverage, if the coverage gets below a certain level, the build should fails.
4. Use PMD, CPD, FindBugs, JSLint, JSHint => Sonar, look for number of warnings, fail the build if it increases.
5. Deploy!

Run one CI per variation of production platform/database/etc.

Rotate CM role to spread risk.

CI to handle legacy code

Make the build fail if the coverage is lower then before. Use other metrics such as warnings, PMD, CPD, FindBugs, JSLint, JSHint and make the build fail if it increases.

Workshop

We finished the course with a workshop where we paired and write the code test first. We started with a strategic design session and the did TDD. After each feature was implemented we switched pairs and got to work on an new code base.

This was our features:

• Check if a given number is a perfect number
• Check if a given number is a abundant number
• Check if a given number is a deficient number

Tools mentioned

Below is the list of different tools and frameworks that was mentions during the course.

Continuous test runner

To have tests running at all changes when doing development. If the runner don’t supports it, write a directory listener that listens for file changes of a specific type and run all tests on them.

• Karma a JavaScript test runner.
• Infinitest plugin for eclipse and IntelliJ that runs all tests on save.

Code coverage

• JaCoCo
• Clover

Try tools such as Guantanamo and Ashcroft to check your coverage in a very evil way.

Java testing

• Spock Written in groovy, uses the given when then pattern.
• JUnitPerf comes in handy to automate performance unit test.
• Jester for mutating your code and exercise your current test suite. It can find missing tests cases.

JavaScript testing

• mocka Testing framework, using the given, when then approach.
• chai Assert framework
• angular-mocks
• JSTestDrive
• NodeUnit for NodeJS (like JUnit)

Database testing

Set the connection in roll-back only mode for the tests.

Tools that can be useful when doing db testing is Liquibase and dbdeploy

For UI testing

• watin For .NET
• watir For Ruby
• watij For Java
• Selenium Testing through a web browser
• Geb Selenium test powerd by a groovy framework
• PhantomJS A headless web browser (based on WebKit)

Functional testing

• Fit
• FitNesse
• easyb
• RSpec
• Cucumber
• SpecFlow

Code analyses

• PMD
• CPD
• FindBugs
• JSLint
• JSONLint
• JSHint

Questions

DDD vs TDD?

DDD is a strategic design, it’s good to concretise some things, but the tactical design should drive for a minimalistic approach. It’s important to have the business people around at all times to mix these two.

More information on ATDD

Books on the topic, Specification by Example by Gojco Adzic.

Integration tests toward the low-level OS functions?

Make sure that the method gets called, use a spy or mock for that or a negative test that is always true in your development environment. For integration tests fire up what environment you need to test on and run the tests there.