Producing Estimates

OK, so it's about time I got back into writing about software development, rather than software (or running, or travelling, or feeds) and the hot topic for me at the moment is the estimation process.

This topic's probably a bit big to tackle in a single post, so it's post series time. Once all the posts are up I'll slap another up with all the text combined.

So – Producing good medium term estimates...

I'm not going to talk about the process for deriving a short term estimate for a small piece of work, that's already covered beautifully by Mike Cohn in User Stories Applied, and I blogged on that topic some time ago. Rather I'm going to talk about producing an overall estimate for a release iteration or module.

I've read an awful lot on this topic over the last couple of years, so I'm sorry if all I'm doing is plagiarising things said by Kent Beck, Mike Cohn or Martin Fowler (OK, the book's Kent as well, but you get the point), or any of those many people out there that blog, and that I read. Truly, I'm sorry.

I'm not intending to infringe copyright or take credit for other people's work, it's just that my thinking has been heavily guided by these people. This writing is how I feel about estimating, having soaked up those texts and put many of their ideas into practice.

Many people think that XP is against producing longer term estimates, but it isn't. It's just that it's not about tying people down to a particular date 2 and a half years in the future and beating them with a 400 page functional design document at the end of it; it's about being able to say with some degree of certainty when some useful software will arrive, the general scope of that software, and ultimately to allow those people in power to be able to predict some kind of cost for the project.

Any business that allows the development team to go off and do a job without providing some kind of prediction back to the upper management team being irresponsible at best, grossly negligent at worst.

Providing good medium term estimates is a skill, and a highly desirable one. By giving good estimates and delivering to them you are effectively managing the expectations of upper management and allowing them to plan the larger scale future of the software in their business. When they feel they can trust the numbers coming out of the development teams it means they are less likely to throw seemingly arbitrary deadlines at their IT departments and then get angry when they're not met. Taking responsibility for, and producing good estimates will ultimately allow you to take control of the time-scales within your department. If you continually provide bad estimates or suggest a level of accuracy that simply isn't there by saying "it'll take 1,203 ½ man days to complete this project", then you've only got yourself to blame when that prediction is used to beat you down when your project isn't complete in exactly 1,203 ½ man days.

Whilst being written within the scope of XP, there's no reason why these practices won't work when you're not using an agile process. All you need to do is to split your work into small enough chunks so that each one can be estimated in the region of a few hours up to 5 days work before you start.

In summary, the process is this:

• Produce a rough estimate for the whole of the release iteration.
• Iteratively produce more detailed estimates for the work most likely to be done over the next couple of weeks.
• Feed the detailed estimates into the overall cost.
• Do some development.
• Feed the actual time taken back into the estimates and revise the overall cost.
• Don't bury your head in the sand if things aren't going to plan.

1 – Produce a rough estimate for the whole of the release iteration.

Before you can start work developing a new release you should have some idea of what functionality is going into that release. You should have a fairly large collection of loosely defined pieces of functionality. You can't absolutely guarantee that these stories will completely form the release that will go out in 3 months time, but what you can say is: "Today we believe that it would be most profitable for us to work on these areas of functionality in order to deliver a meaningful release to the business within a reasonable time-frame".

Things may change over the course of the next few months, and the later steps in this process will help you to respond to those changes and provide the relevant feedback to the business.

Kent Beck states (I paraphrase): You don't drive a car by pointing it at the end of the road and driving in a straight line with your eyes closed. You stay alert and make corrections moment by moment.
I state: When you get in a car and start to drive you should at least have some idea where you're going.

The two statements don't oppose each other.

The basic idea is this:

• In broad strokes, group the stories in terms of cost in relation to each other. If X is medium and Y is huge then Z is tiny.
• Assign a numbered cost to each group of stories.
• Inform that decision with past experience.
  Produce a total.

So, get together the stories that you're putting into this release iteration, grab a room with a big desk, some of your most knowledgeable customers and respected developers and off you go.

Ideally you want to have around 5 to 8 people in the room, more than this and you'll find you end up arguing over details, less and you may find you don't have enough buy in from the customer and development teams.

As a golden rule: developers that are not working on the project should NOT be allowed to estimate. There's nothing worse for developer buy-in than having an estimate or deadline over which they feel that have no control.

Try to keep the number of stories down, I find around 50 stories is good. If you have more you may be able to group similar ones together into a bigger story, or you may find you have to cull a few. That's OK, you can go through this process several times. You can try to go through the process with more, but I find that you can't keep more than 50 stories in your head at one time. You're less likely to get the relative costs right. And it'll get boring.

Quickly estimate each story:

• The customer explains the functionality required without worrying too much about the about the detail;
• The developers place each story into one of 4 piles: small, medium, large and wooooaaaah man that's big.

People shouldn't be afraid to discuss their thinking, though you should be worried if it's taking 10 minutes to produce an estimate for each story.

You may find that developers will want to split stories down into smaller ones and drive out the details. Don't, if you can avoid it. At this point we want a general idea of the relative size of each story. We don't need to understand the exact nature of each individual story, we don't need enough to be able to sit down and start developing. All we need is a good understanding of the general functionality needed and a good idea of the relative cost. Stories WILL be re-estimated before they're implemented and many of them will be split into several smaller ones to make their development simpler. But we'll worry about that later.

As the stories are being added to piles be critical of past decisions. Allow stories to be moved between piles. Allow piles to be completely rebuilt. You may start off with 10 stories that spread across all 4 piles, then the 11th 12th and 13th stories come along and have nowhere to go – they're all an order of magnitude bigger than the ones that have gone before. That's OK, look at the stories you've already grouped in light of this new knowledge and recalibrate the piles.

Once all the stories are done, spread the piles out so you can see every story in each one. Appraise your groupings. Move stories around. It's important that you're comfortable that you've got them in the right pile relative to each other.

If the developers involved in this round of estimating recently worked on another project or a previous release of this project then add some recently completed stories from that work. Have the developers assign those stories into the piles. You know how long those stories took and you can use that to help guide your new estimates. There's nothing more useful in estimating than past experience... don't be afraid to use that knowledge formally.

Once you're that the piles are accurate, put a number of days against each pile. The idea is to get 'on average a single story in this pile will take x days'.

If you have to err, then err on the side of bigger. Bear in mind that people always estimate in a perfect world where nothing ever goes wrong. The stories you added from the last project can be used to guide the number. You know exactly how long it took to develop those stories and you'd expect the new numbers to be similar.

The resulting base estimate is then just the sum of (number of stories in each pile * cost of that pile). This number will give you a general estimate on the overall cost.

You'll probably want to add a contingency, and past experience on other projects will help you guide that. If you've gone through this process before, pull out the numbers from the last release... when you last performed this process and compare the starting estimate with the actual amount of time taken. Use that comparison for the contingency.

The gut feeling is to say, "Hang on, the last time we did this we estimated these 50 stories, but only built these 20 and added these other 30. That means that the starting estimate was nothing like what we actually did". But the point is that if every release has the same kind of people working on it, working for the same business with the same kinds of pressures then this release is likely to be the same as the last. Odds are that THIS release will only consist of 20 of these stories plus 30 other unknown ones... and the effect on the estimate is likely to be similar. Each release is NOT unique, each release is likely to go ahead in exactly the same way as the last one. Use that knowledge, and use the numbers you so painstakingly recorded last time.

Aside: Project managers tend to think that I have a problem with them collecting numbers (as they love to do). They're wrong... I have a problem with numbers being collected and never being used. The estimation process is the place where this is most apparent. Learn from what happened yesterday to inform your estimate for tomorrow.

Once the job's done you should have enough information to allow you to have the planning game. For those not familiar with XP, that is the point at which the customer team (with some advice from the development team) prioritises and orders the stories. It gives you a very good overall view of the direction of the project and a clear goal for the next few weeks of work.

At this point you should have a good idea of cost and scope and have a good degree of confidence in both. Now's the time to tell your boss...

Next up – using the shorter term estimates to inform the longer term one...

Technorati Tags: XP, Extreme+Programming, agile, software, development, programming, estimating, user+stories, software+development, Robert+Baillie