Scrum metrics are a widely covered subject. You can find numerous articles on how to monitor sprint scope, goal, team velocity based on tasks estimated with story points, and interruptions during the sprint, to name a few. But what if you work in an extremely dynamic environment, such as a startup, where changes in directions, reassigning tasks and interruptions can occur every day or even more frequently? In this blog post, we will explain what software development metrics should be taken into account in a startup environment to ensure you keep on top of your project’s progress.
Where Scrum is not enough
The Scrum approach is sometimes enough, but often not for startups, especially those building a new software product. A sprint’s scope and goals can change just one day after they have been agreed. Additionally, with a lot of research tasks it is hard to estimate effort, set up a clear deadline and foresee measurable results. Things that seem easy at first glance turn out to be complex and time-consuming.
So, how do we deal with this, and ensure we’re on the right track? Which agile metrics are most appropriate and give us clear insights into the project? To answer these questions, we’re going to look at three software development metrics that in our experience work best in a startup dynamic environment.
Tasks—how to treat them
When managing startup software engineering projects, story points or other methods of task effort estimation are usually difficult to implement, time-consuming and confusing. Team members, especially at the beginning of the project, have a different understanding of story points, different habits and approaches.
Additionally, time spent estimating is often wasted, as tasks are soon enough withdrawn, frozen or blocked, while others (not estimated during a refinement session) are introduced.
In the ever-changing environment that characterizes startups, Kanban trumps Scrum, and allows you to focus on finishing what has already been started. But to be able to finish, tasks should be short: completing them should take no longer than 2-3 days. So, it is better not to estimate tasks, but to ensure that stories are split into smaller chunks of work, or subtasks. Then we can start measuring.
#1 Throughput—the most important metric
The ability to quickly react to constantly changing demands is essential in the agile approach. But sometimes this principle is understood incorrectly. It is not about immediately starting a new task when it pops up, but finishing what you’ve started and then moving on to your next-highest priorities, while taking capacity into consideration.
In a virtual world, however, we often fall into the trap of stretching our capacity too thin. We start a lot of tasks and then switch between them in an effort to complete them all at the same time. This context-switching not only stretches out the task completion, but causes more mistakes to be made—which take time to fix.
Given all this, the most important metric in the project is the number of tasks actually completed in a given period of time. This is how we should define throughput because ultimately only finished tasks matter. So, stop starting, start finishing!
There might be situations where you have to stop what you’re working on at the moment. This may be true in a production environment, but in software development, especially when we have taks estimated for 2-3 days, it shouldn’t happen.
Interruptions jeopardize the processing of tasks and lower throughput. Controlling interruptions by checking why a given task was stopped should become a habit in a dynamic startup environment.
The number of interrupted tasks in a given period of time is our second metric. Measure it and work on decreasing this value, as interruptions negatively affect your team’s productivity.
#3 Average Cycle Time—the shorter, the better
In the Agile methodology, two time-related metrics are frequently mentioned. The first one is lead time, or the time that elapses from reporting the task to its completion. The second one is cycle time, the time that elapses from starting the task to its completion.
The first metric is often contingent upon the second. When a cycle time is shorter, and you complete tasks quickly and effectively, the queue of pending tasks decreases quickly. A smaller queue means that newly reported tasks are processed faster. The Average Cycle Time is thus very dependent on the Max Time—the time spent on the most time-consuming task. If you do a few tasks quickly, but one takes you too much time, the average time will also be stretched out.
So, stop frustrating yourself with a long list of pending tasks, but focus on splitting them into small chunks properly and finishing what you’ve started as quickly as possible. Smaller tasks are less likely to be interrupted, and bring faster results. The sense of accomplishment you feel when you achieve results will motivate you to move forward. So, measure your Average Cycle Time and do your best to shorten this time.
Dependencies between metrics
Throughput, Interruptions and Average Cycle Time—these three software development metrics are equally important and can tell you a great deal about the progress of your software project. But you can also observe some dependencies between them. Average Cycle Time will not decrease if there are too many Interruptions or Max Time is too long.
Conversely, if Average Cycle Time increases, then Throughput will decrease—and you will have delivered less. If your Throughput and Average Cycle Time are low, you will have completed fewer tasks, but quickly. In an ideal world, of course, you’ll do more and do it quickly.
You can also track trends to build insights and prepare yourself to react quickly to negative developments (see Fig.1).
Fig 1. Delivery metrics
Jira reports to track the metrics
Once we have decided on what to measure, we need to find out how to do it. Jira comes in handy here. The Kanban board with To Do, Blocked, In progress, In review, and Done statuses visualizes your pipeline of processed tasks. You can also set up limits for the number of tasks in a particular column (WIP—work in progress), so the system will help you keep the good practice of finishing open tasks before starting up another.
Throughput is easily measured by the built-in Jira report “Created vs. Resolved Issues Report” (Fig. 2). You can set several parameters there, e.g. periods you would like to check (e.g. weekly or monthly) and how many previous days should be checked. The report shows a comparison of created and resolved tasks in measured periods. The number of resolved tasks is your team’s throughput.
Fig.2 Created vs. resolved tasks in Jira
Interruptions can be checked by filtering tasks which were moved to the Blocked status in the last days. A simple filter JQL (Jira Query Language) command for such filtering is as follows:
status changed to (Blocked) by (username1, username2) after -8d ORDER BY assignee DESC, Rank ASC
You can also use the built-in report “Cumulative Flow Diagram,” which shows changes in a metric’s status in a given period of time.
Average Cycle Time is automatically calculated by Jira and presented on another built-in “Control Chart” (Fig. 3). It looks at all finished tasks. Based on this report, you can also easily analyse deviations from the average execution time. Average time is shown as a red line on the chart. Bear in mind that this average value is calculated based on the entire range of data picked up for the chart.
Fig. 3 Control chart
The three software development metrics I’ve described today are obviously not the only ones you can pick up to measure the progress of your software project. There are a whole bunch of other metrics, like your team’s capacity, number of tasks in backlog, number of defects in a period of time, technical debt, and customer satisfaction. But in our experience, these three are crucial ones in a dynamic startup environment. Other metrics depend on them.
- Split tasks into smaller chunks properly
- Avoid interruptions
- Work quickly on what you have started
- Observe your throughput
- Customer satisfaction will increase
- Backlog, technical debt and number of bugs will decrease
- Your team will be better motivated and happier with the work