What is Stability
Stability is a measure of the volume of requirement change during a sprint expressed as a percentage of change vs plan.
Stability highlights the ability of a team to work with a stable set of requirements during a sprint as we see lower Completion Rates when Stability is lower .
How Umano visualises Stability
How Umano measures this (simplified - see Detailed Modelling here)
Umano measures Stability through calculating:
The number of new tickets created, updated or deleted after starting of the sprint
/divided by
The number of tickets at sprint start
x100
N.B. All Jira fields are weighted equally
A score of 100% means that the sprint was executed as planned without updates to requirements or new work.
When calculating Stability, Umano undertakes the following steps.
Step 1:
Identify the number of tickets that have a time estimate, story point estimate and tickets with no estimate.
Step 2:
Using data from each team over a period of time (usual cycle time (6 intervals)) calculate the average time taken to complete a unit estimate or average time taken to complete a task with similar estimates. When there is no statistic estimate assigned for a ticket, use past similar tickets to predict the estimated time it would take to complete the ticket so that there is a common weighting that is valid for all tickets in the sprint.
Step3:
Use the computed time estimates as weighting calculate the stability impact for each ticket in the sprint.
What’s included?
Included | Not included |
All tickets assigned to the sprint while the sprint is in progress | Sub-tasks within tickets that are inside the sprint Change of labels, related tickets or components |
Practices that influence this measure
Number of tickets removed from planned sprints
Number of tickets added to a sprint in progress
For planned Stories:
Change ratio of Title
Change ratio of Acceptance Criteria
Change ratio of estimated size (story points or time)
Why Stability matters
This is a measure that entirely depends on your team’s context.
A core principle of applying the agile method is that teams welcome change in requirements, even late in development. As such, changing requirements may reflect legitimate shift in priorities, or a response to discoveries that require a change in approach. It can also be reflective of a team’s adaptability in responding to the changing needs of their product manager or customer.
Unstable requirements during a sprint however may be the result of insufficiently planned work. This phenomenon can also be referred to as scope creep and is closely linked to increases in costs and projects going over budget.
Stable requirements provide a team with a stable goal (fixed scope) to work towards. This can build confidence and lead to a team being able to ship features more predictably.
What ‘good’ Stability looks like
Your team’s context is the best arbiter of what ‘good’ looks like for your team.
When a team focuses on improving their speed or progress, higher levels of stability are favoured over lower but 100% stability usually isn't the goal unless you have very predictable work. Use Umano to manage the expectations of your team and peers with a clear eye on the assigned tasks and work with your Product Manager to document and requested changes so they can be reviewed and prioritised when planning for your next sprint.
If your team has prioritised being responsive to customers, lower levels of stability can be expected.
Look out for patterns that don't match your expectations relating to constantly changing requirements over time. This can be reflective of not enough time spent planning, or breaking down stories relating to the sprint goals into smaller, better understood tasks. This pattern can lead the team to feeling like they are ‘spinning wheels’, lacking in a sense of completion and progress. Changing requirements may also have a greater impact on a team the later that the changes are made.
Tips for a more stable sprint
Involve, where possible, the whole team in planning and backlog grooming.
Break stories and task down to smaller tasks.
Discuss dependencies for each tasks.
Invest in stakeholder management to avoid new work being enforced upon the team mid sprint.
Related Articles
How Does Umano handle edge cases in calculating Stability?
If there are only story points in the project/sprint, then weights are handled in story points and tickets with no estimates are given a weight on 1.
This avoids the scenarios where there are smaller-sized story points having longer usual times having a higher impact on Stability
Most of the customer teams, have a usual cycle time for non-estimated tickets that are less than or closer to usual cycle time of story point 1 tickets
If there are both story points and time estimates are assigned to a single ticket, then story points takes precedence as story points consider more dimensions (effort, complexity) than just time estimates.
Detailed Modelling
Step 1:
Identify the number of tickets that have a time estimate, story point estimate and tickets with no statistics estimate.
N = Total number of tickets in the sprint
nt = Number of tickets with time estimates
ns = Number of tickets with story points (or with any other statistic estimate other than time estimates)
nn = Number of tickets without a statistics estimate
Step 2:
Using data from each team over a period of time (usual cycle time (6 intervals)) calculate the average time taken to complete a unit estimate or average time taken to complete a task with similar estimates. When there is no statistic estimate assigned for a ticket, use past similar tickets to predict the estimated time it would take to complete the ticket so that there is a common weighting that is valid for all tickets in the sprint.
T = Time estimated for the whole sprint
t = Time for a single time estimate unit
ut = Number of time estimate units per ticket with time estimates
Tt = Total time for tickets with time estimates
us = Number of time estimate units per story point (or for another unit)
s = Number of story points (or other units) per ticket
Ts = Total time for tickets with time estimates
ue = Number of unit time estimates per ticket without any statistic estimate (predicted using the time taken for similar tickets in the past)
Tn = Total time for tickets without statistic estimate
Estimating time taken per story point/unsized tickets can be done in multiple ways depending on the amount of data available per project/team (as estimation practices can vary from project to project and overtime):
If there are at least over 1000 data points available for each option, then it can be estimated per any or combination of these subcategories:
Issue type
Assignee
The component that the ticket belongs to
Specific words used in the ticket (this probably should be combined with other options unless proven that features generated using the text can be used independently)
If the dataset is not that large, then the estimate can be more generic and consider all the tickets available
Step 3:
Use the computed time estimates as weighting calculate the stability impact for each ticket in the sprint.
Calculate the changes (ch) in each ticket and each change will be weighted by the time estimate C
where u, is the unit time estimate for the ticket
Impact of tickets added and removed will be equal to the time_estimate units that are assigned for those tickets
Notes:
We also disregard the time it takes but assign no estimate tasks a story point depending on the usual cycle time and use story points as weights instead of the time.
This can be useful when there are story points that have irregular usual cycle times.
Example:
Sprint start date: 2019-01-07 10:00:00
Sprint end date: 2019-01-10 17:00:00
Sprint on the first day (2019-01-07):
Ticket_id | Summary | Description | Acceptance Criteria | Story Points | Time Estimates | Issue Type |
XX-0001 | Ticket testing 1 | Test description 1 | Test acceptance criteria 1 | 1 |
| Task |
XX-0002 | Test ticket 2 | Description to test 2 | Test this accept... |
| 1200 | Spike |
XX-0003 | Test ticket 3 | Test description 3 |
| 2 |
| Task |
XX-0004 | Ticket testing 4 | Test description 4 | Test acceptance criteria 4 | 4 |
| Task |
XX-0005 | Ticket testing 5 | Test this description | Test acceptance criteria 5 |
|
| Bug |
Step 1:
nt = 1
ns = 3
nn = 1
(Total number of tickets in the sprint) N = 1+3+1 = 5
Step 2:
t = 1s
Tt = 1200 * 1s = 1200s
us = Assume in the past to complete a story point it has taken 12mins on average
us = 12*60 = 720s
Ts = 720*1 + 720*4 + 720*2 = 5040
Assuming all the bugs belonging to this project with testing in the title has taken 2 hours on average to complete,
ue = 2*60*60 = 7200s
Tn = 7200s
(Total time in the sprint at the start) T = 1200 + 5040 + 7200 = 13440s
Sprint on the second day (2019-01-08):
Ticket_id | Summary | Description | Acceptance Criteria | Story Points | Time Estimates | Issue Type | Updated Time Estimates |
XX-0001 | Ticket testing 1 | Test description 1 | Test acceptance criteria 1 | 1 |
| Task | 720s |
XX-0002 | Ticket testing 2 | Test description 2 | Test acceptance criteria 2 |
| 1200 | Spike | 1200s |
XX-0003 | Ticket testing 3 | Test description 3 |
| 3 |
| Task | 2160s |
XX-0004 | Ticket testing 4 | Test description 4 | Test acceptance criteria 4 | 4 |
| Task | 2880s |
XX-0005 | Ticket testing 5 | Test description 5 | Test acceptance criteria 5 |
|
| Bug | 7200s |
XX-0006 | Ticket testing 6 | Test description 6 | Test acceptance criteria 6 | 1 |
| Task | 720s |
Stability of the sprint on 2019-01-08:
(Test changes are calculated using difflib)
Change in ticket xx-0002:
Changes to acceptance criteria = (Test this accept .... → Test acceptance criteria 2) = 0.4893617021276596
Total changes in xx-0002 = (summary_changes + acceptance_criteria_changes + story_point_changes)*0.33
= (0 + 0.4893617021276596 + 0)*0.33 = 0.16148936170212494
Change in ticket xx-0003:
Changes in story points = (2 → 3) = abs(3-2)/2 = 0.5
Total changes in xx-0003 = (summary_changes + acceptance_criteria_changes + story_point_changes)*0.25
= (0 + 0 + 0 + 0.5)*0.33 = 0.165
Stability change in day 2 = ((Total changes in xx-0002*time_estimate + Total changes in xx-0003*time_estimate)
+ changes due to the addition of xx-0006(time_estimate)
/Total time estimate on the day 1)*100
= ((((0.161489*1200) + (0.165*1440)) + 720)/13440)*100
= 4.283%
Stability of the sprint on 2019-01-08 = 100 - 4.283
= 95.717%
Edge Cases:
If there are tickets added to and removed from the sprint within 30 mins, those sprint changes do not affect the stability calculations.
If there are only story points in the project/sprint, then weights are handled in story points and tickets with no estimates are given a weight on 1.
This avoids the scenarios where there are smaller-sized story points having longer usual times having a higher impact on Stability
Most of the customer teams, have a usual cycle time for non-estimated tickets that are less than or closer to usual cycle time of story point 1 tickets
If there are both story points and time estimates are assigned to a single ticket, then story points takes precedence as story points consider more dimensions (effort, complexity) than just time estimates.