Agile

Technical Debt

We need to start recording our technical debt, I think.

For example, if Identity Server was a packaged third party product we’d be okay but it’s actually quite rough demonstration code with only a small integration test harness. It should be brought up to the same standard as the rest of the code base (eventually). So it works but we have a fair amount of technical debt that we need to record.

Note that we can’t use stories for technical debt. The debt accrued from getting a story to ‘done’ and the points have already been earned.

Also, as we are developing, there will be times we add TODO/HACK into the code but the story still meets its acceptance tests. This extra work should be recorded as tasks in the backlog and then ordered. A rough estimate in hours added to each task will reveal our technical debt.

There will be times when we deliberately let the code quality slip, usually by choosing to ignore some of our metrics going into the red, in order to make a release available. That technical debt needs to be recorded, too.

How we pay down the technical debt is another matter. Preferably we wait until we are revisiting that piece of work and have a need to refactor. If we have so much cruft in a piece of code that adding a new feature is risky then that’s another time when the debt needs to be paid down. Otherwise, let it accrue, debt is a useful resource in the project budget.

Easy to say, harder to do. 🙂

SpecFlow

Acceptance Tests for User Stories allow the Product Owner to more easily say whether or not they accept a Story as ‘Done’. Also, Acceptance Tests can be used in Behaviour Driven Development (BDD) to provide an “outside in” development process that complements the “inside out” coding style of Test Driven Development (TDD).

SpecFlow brings Cucumber BDD to .NET without the need for a Ruby intermediary like IronRuby.

In your Tests project add a Features folder. SpecFlow installs some templates into VS.NET so add a new SpecFlowFeature and fill it out like the following example, InterestRatings.feature :

Feature: Interest Ratings
	In order to manage the Interest Ratings
	As a Trader
	I want an Interest Ratings screen

@view
Scenario: View the Interest Ratings
	Given a repository of Interest Rating records
	When I view the Interest Rating screen 
	Then the full list is created

@create
Scenario: Create an Interest Ratings record
	Given a repository of Interest Rating records
	When I create an Interest Rating with name Test 
	And Interest Rating code 4
	Then the Interest Rating is saved to the repository with 
                 name Test and code 4

The scenarios are the tests. The format is a clear Given-When-Then description. As you create the scenario the .feature.cs will be updated for you.

Now you need to link up the statements in your scenario to steps that the unit test framework can execute. Create a Steps folder under Features and add a SpecFlowStepDefinition. You’ll find the generated file has some useful placeholders to get you started. Here, for example, is InterestSteps.cs :

    [Binding]
    public class InterestSteps
    {
        private IInterestRatingService interestService;
        private InterestRatingViewModel interestRatingViewModel;
        private InterestRating rating;
        private Mock<IValidationService> validationService 
                      = new Mock<IValidationService>();
        private Mock<ILoadScreen> loadScreen 
                      = new Mock<ILoadScreen>();
        private Mock<IServiceLocator> serviceLocator 
                      = new Mock<IServiceLocator>();
        private Mock<IRepository<InterestRating>> interestRepository 
                      = new Mock<IRepository<InterestRating>>();
        private List<InterestRating> interestRatings;

        [Given("a repository of Interest Rating records")]
        public void GivenARepositoryOfInterestRatingRecords()
        {
            Mock<IValidator> validator = new Mock<IValidator>();
            this.serviceLocator
                     .Setup(s => s.GetInstance<IValidationService>())
                     .Returns(this.validationService.Object);
            this.validationService
                     .Setup(v => v.GetValidator(
                         It.IsAny<InterestRating>()))
                     .Returns(validator.Object);
            this.serviceLocator
                     .Setup(s => s.GetInstance<IEventAggregator>())
                     .Returns(new EventAggregator());
            this.serviceLocator
                     .Setup(s => s.GetInstance<ILoadScreen>())
                     .Returns(this.loadScreen.Object);
            ServiceLocator.SetLocatorProvider(() => this.serviceLocator.Object);
            this.interestRatings = 
                      InterestRatingMother
                          .CreateGoodInterestRatingMother()
                          .InterestRatings
                          .Cast<InterestRating>().ToList();
            this.interestRepository
                     .Setup(s => s.GetAll())
                     .Returns(this.interestRatings);
            this.interestService 
                  =  new InterestRatingService(
                                this.interestRepository.Object);
        }


        [When("I view the Interest Rating screen")]
        public void WhenIViewTheInterestRatingScreen()
        {
            this.interestRatingViewModel 
                  = new InterestRatingViewModel(this.interestService);
            this.interestRatingViewModel.Load();
        }

        [When("I create an Interest Rating with name (.*)")]
        public void WhenICreateAnInterestRatingWithName(string name)
        {
            this.interestRatingViewModel 
                  = new InterestRatingViewModel(this.interestService);
            this.interestRatingViewModel.Load();
            this.interestRatingViewModel.Add();
            this.rating 
                  = this.interestRatingViewModel
                              .InterestRatings[
                                   this.interestRatingViewModel
                                        .InterestRatings.Count - 1];
            this.rating.InterestRatingName = name;
        }

        [When("Interest Rating code (.*)")]
        public void AndInterestRatingCode(int code)
        {
            this.rating.InterestRatingCode = code;
            this.interestRatingViewModel.Save();
        }

        [Then("the full list is created")]
        public void ThenTheFullListIsCreated()
        {
            Assert.That(
                this.interestRatings.Count 
                      == this.interestRatingViewModel
                               .InterestRatings.Count);  
        }


        [Then("the Interest Rating is saved to the repository 
         with name (.*) and code (.*)")]
        public void ThenTheInterestRatingIsSavedToTheRepository(
                           string name, int code)
        {
            InterestRating rating 
                = (from m in this.interestRatingViewModel.InterestRatings
                   where m.InterestRatingName.Equals(name)
                   select m).Single();

            Assert.That(
                rating.InterestRatingName.Equals(name),
                "The interest rating name was not saved.");

            Assert.That(
                rating.InterestRatingCode == code,
                "The interest rating code was not saved.");
        }
    }

In particular, notice the reuse of steps, for example GivenARepositoryOfInterestRatingRecords(), and the use of variable placeholders like (.*) to allow the passing of variables into the tests.

BDD wraps TDD. A reasonable flow would be to start with the Story, write up the Acceptance Tests and sketch out some of the steps. As you sketch out the steps you can see what unit tests you need so you go and develop the code using TDD. Once your code is ready and all the unit tests are passing you can integrate the layers with the BDD tests and when those are passing you have fulfilled your Acceptance Test.

Gherkin parsers for SpecFlow are on the way as are VS.NET language plugins (Cuke4VS – currently this crashes my VS.NET 2008).

Cuke4Nuke is another Cucumber port that is worth looking at.

The readability of the Features makes it easy to take Acceptance Tests from User Stories so that the Product Owner and Stakeholders can see what the system is doing. The “outside in” nature of the creating the code gives focus to fulfilling the User Story.

How to fit Scrum into a fixed contract?

We can’t bid on contracts on the basis of doing Scrum as Sales believe they won’t win any contracts. We’re doing Scrum, however, because we believe it increases our changes of delivering. So we’re doing Stealth Scrum in that we won’t explicitly tell the customers about it. We’ve identified a number of ways of creating a contract that can work with Scrum and will be acceptable to our customers.

Time and Materials (T&M) in the terms, perhaps around the installation, can allow for some incremental releases. The bulk of the contract can still be fixed price and the T&M can be budgeted. That way the customer can get sign-off on the budget and we have some room in which to be Agile.

Release points in the contract are so the customer gets early insight into the product. They’ll certainly have feedback and then the functionality may be renegotiated. Done on a capability in, capability out basis the budget need not be changed (which will save a lot of headaches for them as they don’t then need to go and talk to Accounts). If we release Minimum Marketable Features (MMF) early the customer has plenty of opportunity to get involved.

In the contract we talk about how we are going to work with the customer. If regular access to a customer representative can be negotiated then they can be regularly reviewing progress, both catching mis-steps early and also preventing the Team doing work that won’t be accepted or isn’t required. The better communication with the customer will improve the trust between us and them.

Change Requests (CRs) can be created that the customer may, or may not, pay for. Even though the contract will have the functional specification stapled onto the back it’s understood that change must be managed. Stating goals for the project will hopefully guide negotiations around scope creep. CRs are a tricky area as it can damage the trust between ourselves and our customer if not handled sensitively.

We do a Design Study before software construction commences. The customer is guided as to what a Design Study constitutes so that what they expect from it matches closely the coarse-grained estimates you’d expect at this stage. This will minimise the normal contract practices of cost padding and caveats.

Scrum handles what project risk it can through reprioritisation of the product backlog, raising riskier workitems to the top. This needn’t be raised to the customer unless we now suspect something is going all the way to the freezer [front-burner, back-burner, fridge, freezer]. Other project risk has to be addressed through standard project management techniques.

Scrum can manage risk through the trust that is built up between the customer and the team. The customer accepts that the team will do their absolute best to deliver business value as fast as they can and pays for the time required to make the delivery. The team does everything in their power to be transparent so that the customer knows exactly what to expect and can be comfortable that they are getting value. This trust can be difficult to achieve when attempting Stealth Scrum.

Additional Reading.
10 Contracts for your next Agile Software Project
NoFixed.org

Making babies

If we keep metrics of how our Project Velocity changes as we add developers to the Team we can begin to get a rough estimate for how it will affect the Project Burndown.

In this blog post Mike Cohn estimates that adding a seventh developer to a six man team will reduce the velocity by approximately the amount you’d have expected it to rise by in the next iteration. In the second iteration following you will still be below your original velocity and it’s not until the third iteration following that you’ll gain your expected increase in velocity. Note also that the total gain in velocity will be less than ⅙th.

Any developer or project manager instinctively knows this and the reasons have been described clearly. For instance, there are four stages a team goes through: forming; storming; norming and performing [1] (Bruce Tuckman (1965)). More forcefully, there’s Brook’s Law: “Adding manpower to a late software project makes it later.” (or “Nine women can’t make a baby in one month.”).

Danube CSM course

I was lucky enough to be able to attend Angela Druckman’s two day CSM (Certified Scrum Master) course in London this week. I can highly recommend it not least because the trainer was extremely good and very knowledgeable but also because the attendees brought a lot of interesting questions to the discussions.

Danube offers many free resources to help teams implement Scrum:

Story Points vs. Hours for Sprint Burndown

I favour dropping the estimation of Tasks in hours for a Sprint. It’s extra work and it doesn’t add anything to the process.

  • It’s proven that relative estimation is more correct than absolute estimation. Since ideal man days is a time measure it’s all too easy to make absolute estimations.
  • There is no linear correlation between a story point and the number of hours. People who ask how they can correlate story points with ideal time have not yet learned to separate the concepts properly.
  • A story point is a universal measurement across the team. It is not biased by the experience or skills of any individual on the team.
  • In the spirit of continuous improvement Agile practitioners ought to be alert to opportunities to reduce their process overhead (without losing effectiveness).
  • If each story is small enough to be estimated and tested then there may be little benefit achieved through breaking it down into smaller tasks or re-estimating them in hours.
  • The return on investment by estimating in hours is not high enough to make that a serious candidate. We want completed, working software, not very accurate estimates.
  • A Product Owner wants to know how many features the team will deliver not how many hours each of them take. If the Product Owner gets an idea of the relative size he can prioritise and plan accordingly.
  • After the 3rd or 4th sprint, the team reaches a steady state in story points and it becomes easier for the product owner to fill the backlog with story points.
  • It’s difficult and time-consuming to to get developers to estimate time to completion. Developers are terrible at estimating because the work they do isn’t the same each time: either it’s a new business problem or they are dealing with a new technology. Developers also have a tendancy to “hoard” left-over time: if they think they have been allocated so many hours to do something then the work expands to fill the available time.

Dropping hours means that in VersionOne the Sprint Burndown Chart won’t show anything at all but we can refer to the Project Burndown Chart and the Cumulative Flow instead as that burns down Story Points.

Planning Poker

Planning Poker is a fast, card based approach to team estimation. User stories are presented to the team and discussed as a group to round out understanding. In Scrum, this can be played in the Sprint Planning Meeting.

Rules

  1. In the planning session each participant (player) gets a set of cards with the values 0, 0.5, 1, 2, 3, 5, 8, 13, 20, 40, 100, infinity and ?.
  2. One story is estimated at a time.
  3. Each story is discussed briefly.
  4. At the end of the discussion each player selects a card but doesn’t reveal his choice yet.
  5. The cards are all revealed at the same time.
  6. People with high estimates and low estimates are given a soap box to offer their justification for their estimate and then discussion continues. Often it is these discussions that reveal more aspects to be considered.
  7. Repeat the estimation process until a consensus is reached.
  8. If the average is greater than 20 points then the story should be split up and its parts estimated.

Additional Comments

All people who would work on delivery  are involved in estimation.

The cards are numbered as they are to account for the fact that the longer an estimate is, the more uncertainty it contains. Thus, if a developer wants to play a 6 he is forced to reconsider and either work through that some of the perceived uncertainty does not exist and play a 5, or accept a conservative estimate accounting for the uncertainty and play an 8.

Avoid Anchoring where a stated opinion about the amount of time involved in a task can skew the rest of the team’s estimates.

It is important to remember the fidelity of most estimation input data is poor, i.e. we are usually dealing with approximations and best-guesses for work effort. Software development is difficult to predict and we get diminishing returns beyond a certain point of investing more effort in the estimation process.

Resources

There’s a good description of a Planning Poker session by Crisp.
There are free templates for the cards, for example, first page, second page.
The cards can be purchased from various sources but the cheapest are the originals from Mountain Goat Software, Mike Cohn’s company.
For distributed teams there’s a Planning Poker web-based application.