Agile

Technical Debt

We need to start recording our technical debt, I think.

For example, if Identity Server was a packaged third party product we’d be okay but it’s actually quite rough demonstration code with only a small integration test harness. It should be brought up to the same standard as the rest of the code base (eventually). So it works but we have a fair amount of technical debt that we need to record.

Note that we can’t use stories for technical debt. The debt accrued from getting a story to ‘done’ and the points have already been earned.

Also, as we are developing, there will be times we add TODO/HACK into the code but the story still meets its acceptance tests. This extra work should be recorded as tasks in the backlog and then ordered. A rough estimate in hours added to each task will reveal our technical debt.

There will be times when we deliberately let the code quality slip, usually by choosing to ignore some of our metrics going into the red, in order to make a release available. That technical debt needs to be recorded, too.

How we pay down the technical debt is another matter. Preferably we wait until we are revisiting that piece of work and have a need to refactor. If we have so much cruft in a piece of code that adding a new feature is risky then that’s another time when the debt needs to be paid down. Otherwise, let it accrue, debt is a useful resource in the project budget.

Easy to say, harder to do. 🙂

SpecFlow

Acceptance Tests for User Stories allow the Product Owner to more easily say whether or not they accept a Story as ‘Done’. Also, Acceptance Tests can be used in Behaviour Driven Development (BDD) to provide an “outside in” development process that complements the “inside out” coding style of Test Driven Development (TDD).

SpecFlow brings Cucumber BDD to .NET without the need for a Ruby intermediary like IronRuby.

In your Tests project add a Features folder. SpecFlow installs some templates into VS.NET so add a new SpecFlowFeature and fill it out like the following example, InterestRatings.feature :

Feature: Interest Ratings
	In order to manage the Interest Ratings
	As a Trader
	I want an Interest Ratings screen

@view
Scenario: View the Interest Ratings
	Given a repository of Interest Rating records
	When I view the Interest Rating screen 
	Then the full list is created

@create
Scenario: Create an Interest Ratings record
	Given a repository of Interest Rating records
	When I create an Interest Rating with name Test 
	And Interest Rating code 4
	Then the Interest Rating is saved to the repository with 
                 name Test and code 4

The scenarios are the tests. The format is a clear Given-When-Then description. As you create the scenario the .feature.cs will be updated for you.

Now you need to link up the statements in your scenario to steps that the unit test framework can execute. Create a Steps folder under Features and add a SpecFlowStepDefinition. You’ll find the generated file has some useful placeholders to get you started. Here, for example, is InterestSteps.cs :

    [Binding]
    public class InterestSteps
    {
        private IInterestRatingService interestService;
        private InterestRatingViewModel interestRatingViewModel;
        private InterestRating rating;
        private Mock<IValidationService> validationService 
                      = new Mock<IValidationService>();
        private Mock<ILoadScreen> loadScreen 
                      = new Mock<ILoadScreen>();
        private Mock<IServiceLocator> serviceLocator 
                      = new Mock<IServiceLocator>();
        private Mock<IRepository<InterestRating>> interestRepository 
                      = new Mock<IRepository<InterestRating>>();
        private List<InterestRating> interestRatings;

        [Given("a repository of Interest Rating records")]
        public void GivenARepositoryOfInterestRatingRecords()
        {
            Mock<IValidator> validator = new Mock<IValidator>();
            this.serviceLocator
                     .Setup(s => s.GetInstance<IValidationService>())
                     .Returns(this.validationService.Object);
            this.validationService
                     .Setup(v => v.GetValidator(
                         It.IsAny<InterestRating>()))
                     .Returns(validator.Object);
            this.serviceLocator
                     .Setup(s => s.GetInstance<IEventAggregator>())
                     .Returns(new EventAggregator());
            this.serviceLocator
                     .Setup(s => s.GetInstance<ILoadScreen>())
                     .Returns(this.loadScreen.Object);
            ServiceLocator.SetLocatorProvider(() => this.serviceLocator.Object);
            this.interestRatings = 
                      InterestRatingMother
                          .CreateGoodInterestRatingMother()
                          .InterestRatings
                          .Cast<InterestRating>().ToList();
            this.interestRepository
                     .Setup(s => s.GetAll())
                     .Returns(this.interestRatings);
            this.interestService 
                  =  new InterestRatingService(
                                this.interestRepository.Object);
        }


        [When("I view the Interest Rating screen")]
        public void WhenIViewTheInterestRatingScreen()
        {
            this.interestRatingViewModel 
                  = new InterestRatingViewModel(this.interestService);
            this.interestRatingViewModel.Load();
        }

        [When("I create an Interest Rating with name (.*)")]
        public void WhenICreateAnInterestRatingWithName(string name)
        {
            this.interestRatingViewModel 
                  = new InterestRatingViewModel(this.interestService);
            this.interestRatingViewModel.Load();
            this.interestRatingViewModel.Add();
            this.rating 
                  = this.interestRatingViewModel
                              .InterestRatings[
                                   this.interestRatingViewModel
                                        .InterestRatings.Count - 1];
            this.rating.InterestRatingName = name;
        }

        [When("Interest Rating code (.*)")]
        public void AndInterestRatingCode(int code)
        {
            this.rating.InterestRatingCode = code;
            this.interestRatingViewModel.Save();
        }

        [Then("the full list is created")]
        public void ThenTheFullListIsCreated()
        {
            Assert.That(
                this.interestRatings.Count 
                      == this.interestRatingViewModel
                               .InterestRatings.Count);  
        }


        [Then("the Interest Rating is saved to the repository 
         with name (.*) and code (.*)")]
        public void ThenTheInterestRatingIsSavedToTheRepository(
                           string name, int code)
        {
            InterestRating rating 
                = (from m in this.interestRatingViewModel.InterestRatings
                   where m.InterestRatingName.Equals(name)
                   select m).Single();

            Assert.That(
                rating.InterestRatingName.Equals(name),
                "The interest rating name was not saved.");

            Assert.That(
                rating.InterestRatingCode == code,
                "The interest rating code was not saved.");
        }
    }

In particular, notice the reuse of steps, for example GivenARepositoryOfInterestRatingRecords(), and the use of variable placeholders like (.*) to allow the passing of variables into the tests.

BDD wraps TDD. A reasonable flow would be to start with the Story, write up the Acceptance Tests and sketch out some of the steps. As you sketch out the steps you can see what unit tests you need so you go and develop the code using TDD. Once your code is ready and all the unit tests are passing you can integrate the layers with the BDD tests and when those are passing you have fulfilled your Acceptance Test.

Gherkin parsers for SpecFlow are on the way as are VS.NET language plugins (Cuke4VS – currently this crashes my VS.NET 2008).

Cuke4Nuke is another Cucumber port that is worth looking at.

The readability of the Features makes it easy to take Acceptance Tests from User Stories so that the Product Owner and Stakeholders can see what the system is doing. The “outside in” nature of the creating the code gives focus to fulfilling the User Story.

How to fit Scrum into a fixed contract?

We can’t bid on contracts on the basis of doing Scrum as Sales believe they won’t win any contracts. We’re doing Scrum, however, because we believe it increases our changes of delivering. So we’re doing Stealth Scrum in that we won’t explicitly tell the customers about it. We’ve identified a number of ways of creating a contract that can work with Scrum and will be acceptable to our customers.

Time and Materials (T&M) in the terms, perhaps around the installation, can allow for some incremental releases. The bulk of the contract can still be fixed price and the T&M can be budgeted. That way the customer can get sign-off on the budget and we have some room in which to be Agile.

Release points in the contract are so the customer gets early insight into the product. They’ll certainly have feedback and then the functionality may be renegotiated. Done on a capability in, capability out basis the budget need not be changed (which will save a lot of headaches for them as they don’t then need to go and talk to Accounts). If we release Minimum Marketable Features (MMF) early the customer has plenty of opportunity to get involved.

In the contract we talk about how we are going to work with the customer. If regular access to a customer representative can be negotiated then they can be regularly reviewing progress, both catching mis-steps early and also preventing the Team doing work that won’t be accepted or isn’t required. The better communication with the customer will improve the trust between us and them.

Change Requests (CRs) can be created that the customer may, or may not, pay for. Even though the contract will have the functional specification stapled onto the back it’s understood that change must be managed. Stating goals for the project will hopefully guide negotiations around scope creep. CRs are a tricky area as it can damage the trust between ourselves and our customer if not handled sensitively.

We do a Design Study before software construction commences. The customer is guided as to what a Design Study constitutes so that what they expect from it matches closely the coarse-grained estimates you’d expect at this stage. This will minimise the normal contract practices of cost padding and caveats.

Scrum handles what project risk it can through reprioritisation of the product backlog, raising riskier workitems to the top. This needn’t be raised to the customer unless we now suspect something is going all the way to the freezer [front-burner, back-burner, fridge, freezer]. Other project risk has to be addressed through standard project management techniques.

Scrum can manage risk through the trust that is built up between the customer and the team. The customer accepts that the team will do their absolute best to deliver business value as fast as they can and pays for the time required to make the delivery. The team does everything in their power to be transparent so that the customer knows exactly what to expect and can be comfortable that they are getting value. This trust can be difficult to achieve when attempting Stealth Scrum.

Additional Reading.
10 Contracts for your next Agile Software Project
NoFixed.org

Making babies

If we keep metrics of how our Project Velocity changes as we add developers to the Team we can begin to get a rough estimate for how it will affect the Project Burndown.

In this blog post Mike Cohn estimates that adding a seventh developer to a six man team will reduce the velocity by approximately the amount you’d have expected it to rise by in the next iteration. In the second iteration following you will still be below your original velocity and it’s not until the third iteration following that you’ll gain your expected increase in velocity. Note also that the total gain in velocity will be less than ⅙th.

Any developer or project manager instinctively knows this and the reasons have been described clearly. For instance, there are four stages a team goes through: forming; storming; norming and performing [1] (Bruce Tuckman (1965)). More forcefully, there’s Brook’s Law: “Adding manpower to a late software project makes it later.” (or “Nine women can’t make a baby in one month.”).

Danube CSM course

I was lucky enough to be able to attend Angela Druckman’s two day CSM (Certified Scrum Master) course in London this week. I can highly recommend it not least because the trainer was extremely good and very knowledgeable but also because the attendees brought a lot of interesting questions to the discussions.

Danube offers many free resources to help teams implement Scrum:

Development Impediments

The environment at the place I’m currently working is structured to impede development. This can be divided into certain types:

(1) Quality of tools.

The workstations that the developers have are poorly specified for user interface development tasks. The machines are underpowered, resulting in frequent, long timeouts.

The machines cannot be easily upgraded as it is their CPUs that are too long in the tooth. A cost needs to be defined for upgrading machines and when there is the finance available this should be budgeted for. Unfortunately, the company are not big on setting budgets.

It is also difficult to obtain the necessary licences for software.

(2) Control of environment.

The lack of Local Admin rights on the developers’ machines significantly slows the uptake of new technology and the crippled web access severely limits access to information.

The workstation is the developer’s tool of the trade. Restricting the developers’ access to their toolboxes is preventing them from working well. Policies that have been made to prevent abuse of the company’s assets should be enforced through instructions and disciplinary action against offenders; there is no need to treat everyone as if they can’t be trusted to use a computer.

I cannot understate how important this issue is. A software engineer is an expert computer user and there’s no excuse for treating them as anything else.

(3) Specification of projects.

The current projects have no specifications. This means no realistic estimates can be made of when a project will be completed and how much effort will be required. A lot of effort is wasted as the work is developed ad hoc and often requires rework.

Requirements change is to be expected. Indeed, where it provides for a system better aligned with users needs they are to be welcomed. Management, however, must understand and accept the extra costs involved in rework or extra work.

(4) Distrust of developers.

There have been failings from the developers that have resulted, in the past, in misuse of company assets and timewasting. More recently there have been delays in project completion.

To help alleviate this mistrust the developers would like management to be regularly updated on progress. Attention will be drawn to project burndowns and management will be provided with regular demonstrations of the applications.

A company handbook would be useful to state the company’s policies.

Update (Jan 2010)
In the last six months I have made some small progress in clearing these impediments.

(1) The machines have been replaced, the last one today. A lack of planning, however, meant that upgrading a developer’s box, because he’d raised the performance of the old machine as an impediment multiple times, forced the old machine onto a new hire who then had to wait more than two months for his own upgrade.

We currently have developers working on different platforms, unfortunately, with a mix of Windows 7 and XP, and using different IDEs, VisualStudio 2008 and 2010, depending on what works for their environment. There is still no budget set for the purchase of licences.

(2) We have LocalAdmin rights on the boxes now. This has cut some tasks, like upgrading to a new machine, from one week to one day. To get this took raising the issue in almost every meeting with management for the last six months. Developers can now try out new tools and select the most appropriate tools for the job.

(3) Scrum has been implemented as the project management framework and, after a poor uptake, some training was arranged. Requirements are created as User Stories and estimates are made on the basis of Story Points and Velocity. It’s been made clear to management that any other estimates are inaccurate and the team will not be held to them.

(4) Inviting stakeholders to the Sprint Reviews has helped to partially dispel the distrust of the developers.

Management still look down on the developers, fail to understand their motivations and are incautious in revealing their attitudes towards them. Various attempts have been made by management to create some dialogue but most have been abandoned or have failed to appear after their initial announcement. As such, developers are unlikely to approach management when leadership it required.