Agile 42 at the Scrum Gathering

It was great to get together last week with old friends and colleagues Alex Aptus and Andrea Tomasini, who were over in London for the Scrum Gathering 2007. They are both doing Scrum consulting and training in Germany. Andrea's company is called Agile42 and if you think about it the name has some logic to it - "agile" is the undisputed answer to all the major questions of software engineering (mmm...?) in the same way that 42 is the answer to that only slightly larger chestnut: life, the universe and everything. Actually I'd recommend Alex and Andrea's work precisely because they understand that the answers to large questions are not trite or formulaic. If you manage to engage them for your project you'll not only get excellent training in standard methods but thoughtful and experienced consultants that will help you solve the hard questions too.

Another highlight of our meeting was seeing the newly opened St Pancras station on the day that the first Eurostar train arrived there from Paris. The queues at Europe's longest champagne bar meant that we weren't able to order their most expensive bottle at (literally) £2800 a pop!

By the way Andrea has a blog on Software Development Process in Practice. Why not check it out. He has a thoughtful article for example on requirements management. Here's the link: Software Development Process in Practice: Agile Product Development... How to manage Requirements? Effective requirements management has been shown time and again to be the most crucial aspect of successful projects so it's a worthy topic for discussion. I think it's an area that all of us involved with agile processes should be examining. My feeling is that agile methods stop too readily at the level of "features" without also trying to capture other aspects of requirements. 3 Types of Requirement tries to explain this - it summarises the results of some consulting in this area for a client adopting agile methods on very large projects.

Top-down or bottom up planning for Sprints

xProcess supports two ways to specify the required resources for a parent task: top-down or bottom-up.

The simpler approach is bottom-up. A parent task has subtasks with their own estimates of size and effort. The parent task size is therefore just the sum of subtasks' sizes.

What if you don't yet know what all the subtasks are? In this case - or if you want to specify some contingency or slack without inflating the estimates for the subtasks (Parkinson's Law would suggest this is a good idea) - then you can supply an independent estimate for the parent task by specifying it as top-down.

A previous entry, Parent Tasks and Top-down Planning, has already discussed this feature and shows the user-interface for using top-down estimates. The question here (since this is part of the series on Using the pre-defined Scrum Process) is how to use it for Sprints in Scrum.

Let's take a look at the current state of "Sprint 01" in our example Scrum project. Although this is quite a detailed screenshot, just look particularly at the red figures in the middle and the Alerts at the bottom of the screen (click on image to make larger).

The negative numbers for the estimates of size and effort for the "REST" of this task, and the Alerts, tell the same story. The backlog items in this sprint exceed the size of the top-down estimate. (See Prioritising the backlog for how and why these items were added to the Sprint.)

In this state the top-down estimate really is of no use since the larger amount of effort required for the child tasks will determine the time it takes and the top-down estimate of the parent task will effectively be ignored by the auto-scheduler. The auto-fix for the Alerts provides 2 ways to solve this situation:

You can either increase the estimates so that once again the top-down estimate is greater than (or equal to) the sum of the subtasks, or simply convert it to bottom-up so that the estimate for the Sprint just reflects the size of its parts.

So Sprints which already have a complete set of tasks added to them should usually be set to bottom up. However Sprints which do not yet have their backlog items defined should be top-down. That's why when you create Sprints for the first time they are top-down, and as a result you can see the forecast assignments for people and the projected cost of each Sprint of the project, based on its top-down estimate.

The general rule for how to decide whether to use top-down or bottom-up is this:

• if you think that you have captured all the work required in the subtasks, then use bottom-up
• if you think that having completed all the identified subtasks that there may be some others, then use top-down and provide an estimate for how large these other subtasks are in the top-down estimate.
  

These other subtasks are what is referred to in xProcess as the "REST" of a parent task, or the Remaining Essential Sub-Tasks. When you discover them you can create them in the parent task and task-by-task the size of the REST will be reduced, Or you may become confident as time progresses that you have indeed identified all the subtasks so you can then convert the parent task to bottom-up estimating and the REST will be removed.

One more point about parent tasks: team members cannot book their time directly to a parent task, only to a leaf-level task. So if they find that they are carrying out part of the parent task, but not one of the identified subtasks, they may need to create a subtask so they can book to it. This is another way in which the REST of the parent task may be reduced.

The next article on using the Scrum process in xProcess concerns configuring your process with Different Patterns for Backlog Items. Or if you're more interested in how participants on your project can use the xProcess web client, see Participating in the Project.

Prioritising the backlog

The key point about priority-driven planning approaches like Scrum - common in fact to all agile processes - is that the set of tasks undertaken by the team are ordered by priority rather than by a pre-determined procedure. (Priority is usually assessed based on business benefit and risk.) This article focuses on this particular aspect of using Scrum with xProcess and it covers:

• Adding backlog items to specific Sprints
• Showing a Sprint in Burndown and Gantt charts, and finally
  
• Prioritising the backlog

Having created a Scrum project with resources and a set of backlog items (see the previous post in this series) our next step is to prioritise the backlog. Our starting point here is a project which has three Sprints defined and a number of backlog items that have been estimated for approximate size. For simplicity in this example I've created 26 backlog items (with single character names from 'a' to 'z') with sizes of between 1 and 5 "points" (points being nominal units of size corresponding approximately to an ideal day's work for the average worker - see earlier post for more about this topic). Here's a view of my project at this point (click on the image to make it larger).

For the backlog items to be scheduled they must be moved from the "Unscheduled Backlog" task to one of the Sprints. Maybe you have an idea at this stage about the most important tasks, or maybe you want to see how long they would all take given the current estimates and resources. In either case just select the backlog items in the task list and drag-and-drop them to the first sprint ("Sprint 01"). As you'll see in the screenshot, I've opened the Project Explorer pane to do this. It's a view that can be selected from the "Window" menu - or simply click on the project icon next to the project name on the "Tools" pane. Once the backlog items are in a Sprint you'll begin to see the forecast start and end dates for them. You can now hit the "Priorities" button on the Tools pane and begin to put them in priority order. First though let's look at the Gantt chart for "Sprint 01" having added backlog items 'a' to 'v' to it. Double-click on "Sprint 01" to see its details and click on the "Gantt Chart" button. You can view other charts for the Sprint in a similar way, for example by clicking on the "Burndown Chart" button. In the screenshot below you'll also see that I've put the task list in reverse End Date order (by clicking on the column heading) and on the Gantt chart I've scrolled down to show the tasks at the end of the Sprint.

The Gantt Chart shows the forecast start and end dates of the tasks based on resources and estimates so far defined. As time progresses and tasks are started, re-estimated, completed or changed this chart will also change of course - it gives us a view of the likely outcomes based on information currently available. The risk element of the estimates is shown by the 75/95% extensions on the Gantt bars (see "Why I use 3 estimates..." for a discussion of this feature). Given that the target end date for this Sprint is December 19th, we can see that 2 items ('n' and 'a') are in the red zone and not forecast to finish in time and several others are in the amber zone and at risk of not finishing. This would therefore prompt review with the team and the client to whom commitments are being made so that expectations are set appropriately.

Now what if 'n' and 'a' are actually the most important items to get done in this Sprint, or what if there are dependencies between some of the tasks? How are these handled? Well dependencies can be added simply by dragging-and-dropping a task on the task it is dependent on. But it's important for flexibility to keep such dependencies to a minimum. The main way to order tasks in the Scrum process is by priority. To see and change priorities click on the "Priorities" button on the Tools pane. Here's what you might see...

The priorities of all the tasks in "Sprint 01" are all the same (they're all priority one!). As a result the scheduling of the tasks is effectively random. Let's prioritise then in alphabetical order. Here's what the priorities list looks like then...

Changing priority changes the order that the tasks are scheduled, so the Gantt chart now reflects the alphabetical priority ordering. Here's the resulting Gantt chart for "Sprint 01".

And here's the projected Burndown chart for the same Sprint. It still shows a red status as we've not removed tasks by prioritising, but it reflects when the newly ordered tasks are forecast to close. For more on how burndown charts are used as the Sprint progresses, see "Don't burn out - burn down!".

The next article in this series looks at the significance of Top-down or Bottom-up planning for Sprints.

Using the pre-defined Scrum process

I've recently been asked for more guidance about using the pre-defined Scrum process in version 2.9 of xProcess. It uses some of the new features for process definition (like Composite tasks) and also has "top-down" estimating for projects built-in, which is useful for resource planning and financial reporting. So I'm planning to post a series of blog entries on the process and in this one I'll focus on getting started with this process "out of the box". Here's what it covers:

• Creating a Scrum project
• Adding resources to the project
  
• Adding Sprints to the project (a Sprint in Scrum is like a "timebox" in other methods)
• Adding Backlog Items to the project (Backlog Items are the tasks the project must achieve)

So step one is to select the Project Manager perspective (the default) and hit the "New" button so we can make a Scrum project. The New project dialog shows the processes that are available for import, so if you've not already done so this is the point to import "Basic Scrum 2.9.0" and then select "Scrum Project" as your type of project. You can fill in the name and other details of the project (such as the number of 4-week Sprints and a location of a Wiki if you are using one) on this dialog.

At this point take a look at the set of tasks that are already in the project (hitting the "Tasks" button is the quickest way to do this. Here's what you might see (click on the image to make it full size).


There are:

• some overhead tasks (green because they have fixed start and end dates and are within the target dates of the project)
• A "Sprints" parent task containing one sprint "Sprint 01" (these tasks are red since they have NEVER end dates, clearly after the project target)
• An "Unscheduled Backlog" (which is grey as this task is specified as not to be scheduled, also resulting in NEVER end dates).

In the pane below the project details, you can also see there are some alerts displayed telling us that this project needs resources with particular role types. That's a clue to our next step - add resources. Hit the "Resources" button on the "Tools" pane and you'll see an "Add/New Project Resource" button. If you've already defined or imported a set of people you can select those who are going to be on the project, or you can create new persons at this point. Add six team members (Participant role) and one team leader (Scrum Master role) to the project.

At this point we should see actual end dates appearing for the Sprints and the overall project and we can even see a Gantt chart and projected Burndown chart. The Burndown chart is slightly more interesting if we add 2 more Sprints to the project (hit the "New" button and select "Sprint"). Then hit the "Burndown Chart" button on the project pane and you should see something like this:

Now it's important to note that as yet we've entered very little information about this project other than the number of Sprints and the people on the project and their availability. The projected burndown is only reflecting this simple data showing when the work in each Sprint (based on estimates for the amount of work in a Sprint which is modifiable in the process definition or, for a specific Sprint by modifying the estimates in its "Required Resources"). To use Scrum in earnest we need some actual "Backlog Items". Backlog Items are identifiable tasks that in a software development project for example could correspond to features to be built or defects to be corrected. The next step therefore is to add such items.

Hit the "New" button again and select "Backlog Item" (and then "Next"). We have a choice of the 3 Sprints we've defined or the "Unscheduled Backlog" as valid parent tasks for these items. Setting the parent task as Unscheduled Backlog means that we can create a set of tasks without them changing the scheduling of the project. We can then decide later in which Sprint specific tasks should be completed. So with Unscheduled Backlog as the parent task you can give the backlog iten a name and a size (size is measured in "ideal days" by the way), and hit "Finish" and the task will created. If you already have a list of tasks you can copy and paste a comma-separated list into the name field of the "New" dialog, check the "Create Multiple" check-box and several items will be created in one go.

Now we have a project some resources and set of tasks to be completed. The next step is prioritising. We'll look at that in the next posting in this series.

Searching the blog by topic

Blogspot offers some useful facilities for finding blog entries by topic. Below most entries there are a series of labels. Clicking on any of these will bring up other related articles on the same topic. Try it our with the labels below this post.

xProcess Europe

On November 6th Ivis announced that it had demerged its European operation (see Press Release) . The new company is known as xProcess Europe and it's owned by former members of the UK team including Andy Carmichael and Paul Kuzan both seminal thinkers in the evolution of xProcess.

Andy Carmichael: "Paul Kuzan and I are both very pleased to be involved with the on-going evolution of the xProcess product and brand, and we believe xProcess Europe provides a great opportunity, particular for developing the services offering that customers adopting xProcess are often looking for. Providing better support and expertise for companies wanting to improve the processes behind their project and portfolio management will be one of the main focuses of the new company. While our product sales and support work will primarily be targeting the European market, we will continue to offer process improvement services world-wide, wherever the needs exist."

xProcess Europe and Ivis Technologies are both supporters and contributors to this blog. From time to time you'll find links and references to both companies here. For more details try their web sites: www.ivis.com and www.xprocess.eu.com.

The Simple Process... or is it?

The Simple Process in xProcess is the first one you are likely to use. It contains basic patterns for a project, a task a timebox and so on, and it also has some useful actions in there that can be used in your own patterns or even exposed as "User Actions" so they can be called on your projects.

The Simple Process has several elements that you cannot delete as for example xProcess assumes that even if no other Role Types exist (such as Project Manager, Developer, Tester and so on) at the very least the Participant role type will exist. When you define your own process it's likely that you will inherit Simple Process patterns (though you can hide them by defining your own hidden patterns with the same name. So it's definitely worth taking a snoop at all the elements in there. You might even want to cut and paste some bits into your own processes and modify their behaviour.