Stability

View scope change in sprints (items changed, added and removed) mid interval

Chris Boys avatar
Written by Chris Boys
Updated over a week ago

What is Stability

Stability is a measure of the volume of requirement change during a sprint expressed as a percentage of change vs plan.

Stability highlights the ability of a team to work with a stable set of requirements during a sprint as we see lower Completion Rates when Stability is lower .

How Umano measures this (simplified - see Detailed Modelling here)

Umano measures Stability through calculating:

  • The number of new tickets created, updated or deleted after starting of the sprint

    /divided by

  • The number of tickets at sprint start

  • x100

    N.B. All Jira fields are weighted equally

A score of 100% means that the sprint was executed as planned without updates to requirements or new work.

When calculating Stability, Umano undertakes the following steps.

Step 1:


Identify the number of tickets that have a time estimate, story point estimate and tickets with no estimate.

Step 2:


Using data from each team over a period of time (usual cycle time (6 intervals)) calculate the average time taken to complete a unit estimate or average time taken to complete a task with similar estimates. When there is no statistic estimate assigned for a ticket, use past similar tickets to predict the estimated time it would take to complete the ticket so that there is a common weighting that is valid for all tickets in the sprint.

Step3:


Use the computed time estimates as weighting calculate the stability impact for each ticket in the sprint.

What’s included?

Included

Not included

All tickets assigned to the sprint while the sprint is in progress

Sub-tasks within tickets that are inside the sprint

Change of labels, related tickets or components

Practices that influence this measure

  • Number of tickets removed from planned sprints

  • Number of tickets added to a sprint in progress

  • For planned Stories:

    Change ratio of Title

    Change ratio of Acceptance Criteria

    Change ratio of estimated size (story points or time)

Why Stability matters

This is a measure that entirely depends on your team’s context.

A core principle of applying the agile method is that teams welcome change in requirements, even late in development. As such, changing requirements may reflect legitimate shift in priorities, or a response to discoveries that require a change in approach. It can also be reflective of a team’s adaptability in responding to the changing needs of their product manager or customer.

Unstable requirements during a sprint however may be the result of insufficiently planned work. This phenomenon can also be referred to as scope creep and is closely linked to increases in costs and projects going over budget.

Stable requirements provide a team with a stable goal (fixed scope) to work towards. This can build confidence and lead to a team being able to ship features more predictably.

What ‘good’ Stability looks like

Your team’s context is the best arbiter of what ‘good’ looks like for your team.

When a team focuses on improving their speed or progress, higher levels of stability are favoured over lower but 100% stability usually isn't the goal unless you have very predictable work. Use Umano to manage the expectations of your team and peers with a clear eye on the assigned tasks and work with your Product Manager to document and requested changes so they can be reviewed and prioritised when planning for your next sprint.

If your team has prioritised being responsive to customers, lower levels of stability can be expected.

Look out for patterns that don't match your expectations relating to constantly changing requirements over time. This can be reflective of not enough time spent planning, or breaking down stories relating to the sprint goals into smaller, better understood tasks. This pattern can lead the team to feeling like they are ‘spinning wheels’, lacking in a sense of completion and progress. Changing requirements may also have a greater impact on a team the later that the changes are made.

Tips for a more stable sprint

  • Involve, where possible, the whole team in planning and backlog grooming.

  • Break stories and task down to smaller tasks.

  • Discuss dependencies for each tasks.

  • Invest in stakeholder management to avoid new work being enforced upon the team mid sprint.

Related Articles

Frequently Asked Questions

How Does Umano handle edge cases in calculating Stability?

  1. If there are only story points in the project/sprint, then weights are handled in story points and tickets with no estimates are given a weight on 1.

    1. This avoids the scenarios where there are smaller-sized story points having longer usual times having a higher impact on Stability

    2. Most of the customer teams, have a usual cycle time for non-estimated tickets that are less than or closer to usual cycle time of story point 1 tickets

  2. If there are both story points and time estimates are assigned to a single ticket, then story points takes precedence as story points consider more dimensions (effort, complexity) than just time estimates.

Detailed Modelling

Step 1:

Identify the number of tickets that have a time estimate, story point estimate and tickets with no statistics estimate.

N = Total number of tickets in the sprint

  1. nt = Number of tickets with time estimates

  2. ns = Number of tickets with story points (or with any other statistic estimate other than time estimates)

  3. nn = Number of tickets without a statistics estimate

Step 2:

Using data from each team over a period of time (usual cycle time (6 intervals)) calculate the average time taken to complete a unit estimate or average time taken to complete a task with similar estimates. When there is no statistic estimate assigned for a ticket, use past similar tickets to predict the estimated time it would take to complete the ticket so that there is a common weighting that is valid for all tickets in the sprint.

T = Time estimated for the whole sprint

  1. t = Time for a single time estimate unit

  2. ut = Number of time estimate units per ticket with time estimates

  3. Tt = Total time for tickets with time estimates

  4. us = Number of time estimate units per story point (or for another unit)

  5. s = Number of story points (or other units) per ticket

  6. Ts = Total time for tickets with time estimates

  7. ue = Number of unit time estimates per ticket without any statistic estimate (predicted using the time taken for similar tickets in the past)

  8. Tn = Total time for tickets without statistic estimate

Estimating time taken per story point/unsized tickets can be done in multiple ways depending on the amount of data available per project/team (as estimation practices can vary from project to project and overtime):

  • If there are at least over 1000 data points available for each option, then it can be estimated per any or combination of these subcategories:

    • Issue type

    • Assignee

    • The component that the ticket belongs to

    • Specific words used in the ticket (this probably should be combined with other options unless proven that features generated using the text can be used independently)

  • If the dataset is not that large, then the estimate can be more generic and consider all the tickets available

Step 3:

Use the computed time estimates as weighting calculate the stability impact for each ticket in the sprint.

  1. Calculate the changes (ch) in each ticket and each change will be weighted by the time estimate C

    where u, is the unit time estimate for the ticket

  2. Impact of tickets added and removed will be equal to the time_estimate units that are assigned for those tickets

Notes:

We also disregard the time it takes but assign no estimate tasks a story point depending on the usual cycle time and use story points as weights instead of the time.

This can be useful when there are story points that have irregular usual cycle times.

Example:

Sprint start date: 2019-01-07 10:00:00

Sprint end date: 2019-01-10 17:00:00

Sprint on the first day (2019-01-07):

Ticket_id

Summary

Description

Acceptance Criteria

Story Points

Time Estimates

Issue Type

XX-0001

Ticket testing 1

Test description 1

Test acceptance criteria 1

1



Task

XX-0002

Test ticket 2

Description to test 2

Test this accept...



1200

Spike

XX-0003

Test ticket 3

Test description 3



2



Task

XX-0004

Ticket testing 4

Test description 4

Test acceptance criteria 4

4



Task

XX-0005

Ticket testing 5

Test this description

Test acceptance criteria 5





Bug



Step 1:

nt = 1

ns = 3

nn = 1

(Total number of tickets in the sprint) N = 1+3+1 = 5

Step 2:

t = 1s

Tt = 1200 * 1s = 1200s

us = Assume in the past to complete a story point it has taken 12mins on average
us = 12*60 = 720s

Ts = 720*1 + 720*4 + 720*2 = 5040

Assuming all the bugs belonging to this project with testing in the title has taken 2 hours on average to complete,

ue = 2*60*60 = 7200s

Tn = 7200s

(Total time in the sprint at the start) T = 1200 + 5040 + 7200 = 13440s



Sprint on the second day (2019-01-08):

Ticket_id

Summary

Description

Acceptance Criteria

Story Points

Time Estimates

Issue Type

Updated Time Estimates

XX-0001

Ticket testing 1

Test description 1

Test acceptance criteria 1

1



Task

720s

XX-0002

Ticket testing 2

Test description 2

Test acceptance criteria 2



1200

Spike

1200s

XX-0003

Ticket testing 3

Test description 3



3



Task

2160s

XX-0004

Ticket testing 4

Test description 4

Test acceptance criteria 4

4



Task

2880s

XX-0005

Ticket testing 5

Test description 5

Test acceptance criteria 5





Bug

7200s

XX-0006

Ticket testing 6

Test description 6

Test acceptance criteria 6

1



Task

720s

Stability of the sprint on 2019-01-08:

(Test changes are calculated using difflib)

Change in ticket xx-0002:

Changes to acceptance criteria = (Test this accept .... → Test acceptance criteria 2) = 0.4893617021276596

Total changes in xx-0002 = (summary_changes + acceptance_criteria_changes + story_point_changes)*0.33

= (0 + 0.4893617021276596 + 0)*0.33 = 0.16148936170212494



Change in ticket xx-0003:



Changes in story points = (2 → 3) = abs(3-2)/2 = 0.5

Total changes in xx-0003 = (summary_changes + acceptance_criteria_changes + story_point_changes)*0.25

= (0 + 0 + 0 + 0.5)*0.33 = 0.165



Stability change in day 2 = ((Total changes in xx-0002*time_estimate + Total changes in xx-0003*time_estimate)

+ changes due to the addition of xx-0006(time_estimate)

/Total time estimate on the day 1)*100

= ((((0.161489*1200) + (0.165*1440)) + 720)/13440)*100

= 4.283%

Stability of the sprint on 2019-01-08 = 100 - 4.283

= 95.717%

Edge Cases:

  1. If there are tickets added to and removed from the sprint within 30 mins, those sprint changes do not affect the stability calculations.

  2. If there are only story points in the project/sprint, then weights are handled in story points and tickets with no estimates are given a weight on 1.

    1. This avoids the scenarios where there are smaller-sized story points having longer usual times having a higher impact on Stability

    2. Most of the customer teams, have a usual cycle time for non-estimated tickets that are less than or closer to usual cycle time of story point 1 tickets

  3. If there are both story points and time estimates are assigned to a single ticket, then story points takes precedence as story points consider more dimensions (effort, complexity) than just time estimates.

Did this answer your question?