Sprint Tester

fig3_sprint

A High Confidence Sprint

fig4_sprint

A Risky Sprint

What to test our beta version?

A sprint plan consists of a list of stories or development work items and a time box for completing the work items. To establish the plan, the plan needs to know a sizing for each story, and the team velocity, the size of the effort the team is able to in a given sprint. In agile the sizing is often given in story points and the velocity is given in story points per sprint. So a creating a sprint plan entails finding a set of stories in the backlog that:

 

  1. That have the highest priority, and
  2. Whose total story points sum up to the velocity.

 

This sounds fairly straight-forward. However, both the size and the velocity are not known for sure. Both the story sizes and velocity are educated guesses fraught with uncertainty.

So, like it or not, sprint planning involves making an estimate of what you think you your team can complete. Not only that, but, when committing to the plan, you are publically betting your team can and will complete the sprint. The seriousness of this bet depends on the organization. However, most development organizations have stakeholders who are counting on timely delivery and missing a lot of sprints is not OK.  (We will be releasing a release tester soon).

There are various techniques for assigning sizing stories. A common and effective method is called planning poker. These techniques involve each of several team members ‘bidding’ the number of points to the story. Often, the bids are restricted to the Fibonacci numbers, but this isn’t required. Then typically, the team selects one of the bid values, throwing out the extreme values and choosing a single consensus value.

The problem is that this method throws out useful information: For each of the bids, someone who is entitled to an opinion might think the final bid is too high or too low and there is a non-zero probability they are right. To capture this insight, it would be better to capture and use the low, high, and consensus likely value. This could be the majority bid or the average of the bids.

 

For that reason, the Aptage release tester takes as input three values of the story points for each story.

  1. Low or best case
  2. High or worse case
  3. Expected some number between the best and worst case.

 

With this information, the team is saying they size the story to be definitely between the low and high, with the most likely sizing being the expected case. This is the information used by the Aptage tester. In Aptage, you can set the low and expected to be the same value or the expected and high to be the same, or all three the same.

Some think that entering three numbers is burdensome, but in fact agreeing on three numbers is often more burdensome. In any case, the only restriction is that for each story :

 

low ≤ expected ≤ high

 

The team velocity is also uncertain. This is discussed in detail in (link to velocity learner)

 

In The Aptage sprint tester tells you the likelihood of successful completion of the sprint based on:

  • Your recent velocity history (like the learner)
  • The low, expected, high number of story points for each included story.

 

Here is an example of the input screen for the Sprint Tester.

fig1_sprint

Figure 1 The Apatage Sprint Test input screen.

 

With these entries, the output is

fig2_sprint

Figure 2 The likelihood of completing on time

 

The output is a probability distribution of completing on time. The x axis the future date and the y axis is the probability of the competing on the date. The vertical line is the target date, the green area is on time or early and the red area is after the target date.

 

The area under the whole curve is 1.0. The fraction of that area in green is probability of shipping on time. As you can see, the likelihood of completing the plan is XXX%.

 

If sprint tester is mostly green, the plan is feasible and a good bet.

 

fig3_sprint

Figure 3 A low risk plan

 

On the other hand, if the plan is mostly red, the plan is risky and has a high likelihood of failure.

fig4_sprint

Figure 4 I high risk plan

How much risk you plan to assume depends on your risk appetite or perhaps you have some reason to believe an aggressive plan will succeed and are willing to make the bet.

 

The math behind is very similar to the math used for election prediction in Nate Silver’s FiveThirtyEight prediction polls. You will see some very similar displays. For a more detailed description of the math see Sprint Tester Math Notes.

What to test our beta version?