What's the quickest way to make a Gantt chart from an agile board?

Don't say it! I know. :-)

But someone actually asked this question this week, adding presumably to avoid embarrassment that the purpose was to "translate reality into management artifacts". He has a point of course. Familiar artifacts can allow people to see new things more clearly; we just also need to ensure that we've all understood what is really different in the new world.

Any way, I suggested looking at the Open Source tool xProcess, which uses its built-in forecasting engine to put start and end dates on a hierarchical collection of tasks or work items based on size estimates and team availability. I reckon if you have a table with task names you can generate such a chart in about 5 minutes.

Here are the instructions. Download xProcess from this site: http://sourceforge.net/projects/xprocess/

1. Create a project by hitting the "New" button like this:

You can choose various templates but just choose Simple Process / Simple Project. It's the.. well simplest.

2. Add the list of work items by selecting "New" again and "Task". (See next screenshot).

3. Hit "Next" and you can add all the names from the cards on your wall as a comma separated list (see screenshot on the right). Leave the "Size" as 2 - we'll come back to that.

4. Now when you go to the Task List tab (or hit  the "Tasks" button) you should see something like the screenshot below. All your tasks are listed all of size 2.0 and with start and end forecast dates of NEVER.

In other word the Scheduler is forecasting none of these task will finish. That's because we have no resources assigned to the project. The simplest quick fix for this is to add yourself to the project. So...

5. Hit the "Resources" button and then "Add/New Resource", select yourself (or add team members) and then hit "Next". This gets to the "Appointment to Project" dialog shown below. You can define availability on this screen or just hit "Finish".

We now have enough information to generate a Gantt chart, albeit with some fairly big assumptions such as a uniform size of tasks and 100% availability of the team we've assigned. Nevertheless we can hit the "Gantt Chart" button and see the result.

Here's my attempt.

There are lots of things you may wish to adjust at this point like the priority order of the tasks, the estimates for the items (either the default value or individual ones) and team availability. You can even add separate subtasks for different parts of the process into a task template and add team roles so the right people are assigned to the right types of subtask. There are lots of adjustments you can make depending how much longer than 5 minutes you want to spend on this. You can even enter (or generate) fixed "target" dates which compare the forecast dates with dates that may have been agreed with a customer ( or are simply the forecast from last month). Endless hours of fun are possible. :-)

However if you just want to give your manager a comforting diagram, the steps above could be 5 minutes well spent!

"What's your favourite technique for estimating stories in Scrum?"

My number one favourite? Not estimating them at all! 

Instead, ensure stories do not exceed an agreed size that ensures they fit easily within a sprint (ok - I agree that's estimating, but it's much simpler estimating than tee-shirt sizes (S, M. L or/and too big) or points (1, 2, 3, 5, 8,  and too big). Then spend the time you've saved estimating the size of stories on estimating the number of stories in the various epics in your product backlog. Measure velocity in stories not points (or use only two sizes: 1 point and too big) and forecast the completion of epics and minimum-marketable-features based on the throughput of stories (velocity) and the number of stories. 

As well as saving a ton of time, it turns out your forecasting will be just as accurate as if you'd spent ages agonizing over each story! See for example +Vasco Duarte's blog about the evidence for improved forecasts from this approach.

The key question - as +Neil Killick points out in his recent "People Need Estimates" - is what problem are we trying to solve with our estimating? 

I was working with a team recently who were concerned that they had missed their forecast for the second sprint in a row. The retrospective meeting came up with several suggestions about how they could improve their forecast for the next sprint. I asked the team whether there was anyone (outside the team themselves) who was interested or concerned about their forecasting ability. The last few sprints were running at a throughput that was over double the average for the previous year. If the team continued to focus on the improvements they were making, the business could easily adjust to this higher efficiency. The business did not need (or probably believe) the over-optimistic forecasts that the team were making. Shown the evidence of actually completed work, the business will adjust to the new opportunities the improvements open up.

Story Points Considered Harmful

Here's a really interesting and thoughtful discussion of story point estimating by Vasco Duarte: Software Development Today: Story Points Considered Harmful - Or why the future of estimation is really in our past.... He's just followed it up (Software Development Today: A better way to predict project release date!) with analysis from one project which shows that forecasts based on counting stories were more accurate than forecast based on story points. I'd like to see more data, which hopefully will be forthcoming from other projects submitting data, but given the cost of estimating - whether using story points or other metrics - these results provide a very good basis for forecasting in a much simpler way. I look forward to more on this in the future.

Precision and accuracy in estimating

The difference between precision and accuracy has always been important to engineers and it is vital to understand too in the context of estimating for software projects. One approach to estimating a project is to create a detailed work breakdown structure and estimate each task, say in person-hours. A friend told me recently she had consulted on a project which had estimated its size as 135,213 person-hours! The precision in this case is 6 significant figures. As to the accuracy however, previous projects had errors or between 10 and 200%. The precision actually gets in the way of understanding the estimate, particularly as crucial business decisions involving significant proportions of a company's revenue may be based on such numbers. While three-point estimating (3pe) provides one way of capturing risk estimates along with the estimate themselves, the main contribution of agile methods in this area is the tight feedback loop from measured velocity timebox by timebox. Check out some of the links below to see how xProcess supports these concepts.

In search of a definition of uncertainty in three point estimates

I've written elsewhere in this blog about the value and theory of three-point estimating (3PE). See "3PE - why I use three estimates where one might do!". The main practical snag with using 3PE in an agile context is the additional overhead of thinking of 3 numbers every time you want to confirm "effort to complete" - ideally something people can quickly update even on a daily basis.

So I'm working on a mechanism which shows the level of uncertainty in an estimate to complete which takes into account the effort completed to date (T) and the three points of the estimate: best case (b), most likely (m) and worst case (w). If you're interested in the intricacies of such things, please read on!

The composite estimate (E) is the estimate is the derived "median" which is used when one number is required (as for example in the "Total" field in the screenshot above). It is derived from best case (b), most likely (m) and worst case (w) as follows based on assumptions about distribution of cases between best and worst:

E = (b + 4m + w) / 6

Various formulae are possible for our level of uncertainty or estimate risk (R). Here's a starting point expressed in terms of just b and w as follows:

R = (w - b) / (w + b - 2T) ..................................[1]

This expresses the average "error", (w-b)/2, as a proportion of the average time to complete, (w+b-2T)/2. However since it ignores the most likely estimate, it doesn't take into account that for example the worst case may be much further away from the most likely than the most likely is from the best case. An alternative formula would therefore be:

R = (w - m) / (m -T) ........................................[2]

But this formula ignores the best case completely. One could say the "error" should be defined as whichever is greater out of (w-m) or (m-b), but actually it's not this aspect that worries me practically. The worst case is always more significant from a forecasting viewpoint, and from an estimating viewpoint it is the one that can be much further in error than the best case, which can never go lower than T and in practice will always be a little bit higher than T (or you can forget about estimating and just finish it!).

So a better approach is to define the "error" in the formula for R relative to the median estimate, E, all three points are then taken into account. It's much more satisfactory.

Following this approach, here's the formula for the uncertainty in the estimates (estimate risk) that I'll be using:

R = (w - E) / (E - T) ........................................[3]

For those interested in the mechanisms within xProcess to use this formula, there'll be a discussion on that project's wiki about how users can set estimate to complete and level of uncertainty rather than worrying about 3pe every time they update time to complete.

Searching the blog by topic

Blogspot offers some useful facilities for finding blog entries by topic. Below most entries there are a series of labels. Clicking on any of these will bring up other related articles on the same topic. Try it our with the labels below this post.

Set size to match effort

I explained in another entry that size and effort are distinct metrics for tasks. However it's useful to keep the overhead of estimating two closely related items to an absolute minimum. This is why the patterns in the "Simple Process" uses the user's input of a size metric (in "ideal days") to automatically complete the effort estimates (usually displayed in person-hours or person-days). They can be changed subsequently though and if you want to re-synchronize them at some point - say when you a re baselining a plan - a utility for doing so is useful.

Here's a simple Action that you can define in your process (make it a "UI Action" by setting this field to True).

Action Name: Set size to match effort
Expression: setSize(getEffortIncludingChildren()/480.0)
Applicable to: Task
UI Action: True
Parameters: (none)

Having done this you can expand your task hierarchy in the explorer, select the tasks you wish to set size on, and then right-click and select UI Action -> Set size to match effort .

Note: From version 2.9 of xProcess this action is included in the Simple Process.

Great resources for estimating

There are two new books on estimating that I've been finding very useful recently, and one older one. The older one first; and here you understand I'm not referring to Larry Putnam Senior himself! It's his book, co-authored with Ware Myers, "Measures for Excelence: Reliable software on time on budget" that remains a classic, full of helpful advice and packed with actual measures from real projects. I met Larry in 1992 just after his book came out, when I was in Washington with Ed Yourdon presenting some research on software reuse from Japan. He kindly gave me a signed copy and it has been in frequent use since that time, and was a key source for my own book "Better Software Faster" published in 2002. Putnam and Myers' latest book "Five Core Metrics" is another excellent resource which highlights the essentials that all software projects should measure, with practical information about how to do it. In case you're wondering the five are:

• Time
• Effort
• Quality
• Productivity
• Size

While Putnam and Myers experience covers projects ranging from the small to the truly massive, using a wide range of methodologies and covering five decades of software projects, Mike Cohn's book "Agile Estimating and Planning" is focused very much on the agile development processes of more recent history. It is similarly an excellent read with a mass of very practical advice for project managers of agile projects. I particularly liked the "Planning Poker" variant of the Wideband Delphi approach (!) but there are many other practical suggestions and insights, including discussions of burndown charts, velocity and prioritisation.

The challenge for me, working as I am on the product direction of xProcess, is to ensure that the full range of techniques and processes represented in these books is well supported by the underlying model and user interface of the product. xProcess has always been about agility, and in particular bringing powerful techniques from established project management methods and making them available to managers and participants of agile projects in such a way that they remain agile. These books similarly contribute to that drive to achieve controlled, productive and agile software projects.

Estimating size and effort - Why are they different?

A common question I get asked about estimating in xProcess (after three-point estimating of course, which is another popular question!) is: "What's the difference between a task's estimate for size and its (three-point) estimate for effort... and why do I need both?".

There are several ways to answer this, but perhaps a starting point is to ask why we think they're the same. If I ask you the size of your swimming pool you're more like to give me an answer in feet or metres that to reply with the number of person days it took to build the pool. (Building swimming pools is an xProcess application by the way - but that's another story!) When we are discussing the planning of a project the first thing to consider is the size (in whatever units are appropriate) of the deliverables of that project. Then we need a function (based on previous experience) to map from size to the estimate of the effort and consumables required to provide those deliverables.

Because the size of the things we want to deliver are measured in many different units, all of which we want to map eventually to an estimate of effort, it is common for projects to estimate directly in effort units such as person-days, person-hours or person-years. However if we do this, we lose a very important measure of the effectiveness of the project. Questions of team velocity or productivity become somewhat meaningless (except as a correction factor on original estimates of effort, compared to actual effort expended).

So if we can agree size is an important metric for projects, the next question is what units should it be estimated in, and can we compare the estimates with actuals at the end of the project. Software projects in particular are notoriously difficult to find an appropriate measure for size. Lines of source code is commonly used (because they are easy to count) but they don't readily map either to effective functionality delivered nor to effort required to develop them. Function points is another option and more recently "story points" (for an XP user story) or simply "points" have gained popularity where these are arbitrary units that are comparable only within the single project where they are being used. Another useful measure is "ideal-days" - in other words the amount that can be produced by a typical team member in a completely uninterrupted day. We rarely get such days (if ever) so it will take most of us more than a day to implement an ideal-day's worth of size. Nevertheless it is an understandable and reasonably verifiable measure of size.

In xProcess the units of size are common to the whole Data Source (they can be set in Data Source Preferences) and the value is used in several important places. When creating tasks the size value is often used in process patterns to set initial values for three-point estimates. The size units are used in burndown charts and earned-value plots to show progress of a project towards creating its deliverables. And size is also used to calculate productivity measures such as team velocity and software productivity.