Hello everyone, Pavlo Kondratiev from DK Yuztech is on the phone again. I continue to work in a product team developing b2b applications, and on the horizon last year, the Customer and I came to the question – how do we measure the performance of our team and identify weak points in the processes to make development more efficient?

I suggest you familiarize yourself with our experience in implementing team performance measurement metrics.

1. The essence of the problem or the differences between product and commissioned development

Product development differs from recommended development because recommended development is based on Projectwhich by definition is unique within the framework of the customer’s tasks and finite in nature, while at the core of the product the main ones are Value and Financial model.

As for the Project, I made the project — the project is closed. Yes, service support, creation of new functionality, etc. is possible. But regardless of the type of payment, Fix Price or Time Material, which are applied based on project conditions, it remains within the framework of service support or other projects within the framework of the implemented functionality. And, most importantly, the main decision about what will still be in the product is the vision of only the Customer.

In the case of the Product, the situation is slightly different. When a Product is launched, it is assumed that it will solve someone’s problems, and there is a certain financial model built into it, which will always affect the final functionality. That is, the final goal and vision of what the product will look like as a result is not a priori at the start, it cannot be calculated in advance. The definition of what will lie in the further changes of the Product will be determined by hypotheses that will go through the stages of research, will be overgrown with financial metrics that should justify them, because we remember:

Any new functionality in the Product should not just solve some task, it should carry Value that will generate financial improvements and should favorably affect the financial model in one way or another.

Thus, the Product has no final purpose.

More importantly, the product team is often the source of planned change. Yes, it should be based on hypotheses, have financial justification and that’s it. The planned changes may have business customers external to the product team, but this is a long game with endless changes as long as it is justified by the financial model. For this, there is even a separate role in the product team and the person responsible for forming hypotheses that will form the basis of future development is the Product Owner. A person who is at the intersection of business, market and development.

If in the Project the success of the final goal is decided by entering into the classic project triangle, which is based on Scope, Resources and Time, then in the Product a lot will depend on the success of improving the characteristics of the Financial Model. This is not to say that the design triangle is not important, but considering it becomes an unbearably time-consuming process, which automatically leads to a slower pace of delivery of new features, which clearly will not have a beneficial effect on the Product’s journey and its financial model. The classic Waterfall with the existing mandatory stage of collecting and formalizing all requirements at the initial stage will take up too much precious time. The resulting requirements will most likely change, so Waterfall is not a good fit here. Only iterations, testing of hypotheses before the start of development and after the release of a Feature, and new packs of hypotheses in order to understand whether the Product is on the right track, allow you to develop the product quite flexibly, responding to challenges and reversals that arise for the purpose of development.

The question indicated in the announcement arises: what to do then? How to calculate time and resources? How to measure team performance and understand what needs to be fixed?

2. Time to Market

This is a metric that came to IT from business. The first Time2Market conference was held by Bart Hall in October 1995.

Time to Market is a key metric used to assess the effectiveness and speed of bringing a new product, service or functionality to market. It measures the time interval from the moment an idea is fixed to the moment the product is launched on the market or accessed by end users.

That is, this time, which includes both full-fledged research of the hypothesis, and the Discovery phase, in which architectural solutions are fixed, analytics are collected, layouts for functionality are drawn, and the Delivery phase, within which tasks are taken up by developers, checked for quality by QA engineers and the result of the development is put into industrial operation in a productive environment.

3. Lead Time

This is part of Time2Market, which excludes research and hypothesis testing from the Discovery cycle. Exceptional team effort – Discovery + Delivery:

4. Cycle Time

This metric is part of both Lead Time and, accordingly, Time to Market, and shows only the temporary costs of the Delivery phase from the start of development to release:

5. So how do you track and interpret metrics?

Any development will require reporting, which means the product team will inevitably use any task tracker. In order to synchronize within the team, the tasks have a set of statuses, otherwise it is impossible to understand at what stage the work is currently underway. Will it just be changing statuses or moving cards around the Kanban board? It doesn’t matter which methodology you use for calculating metrics – SCRUM or Kanban – only one thing is important. Speed ​​of movement by statuses.

We will also assume as an axiom that the maturity of the processes and your common sense led you to the fact that before starting the development of the task, it is necessary to decompose and evaluate it.

Let’s go from the bottom up and look at Cycle Time. The status of a task in a production process will inevitably be tied to the responsibility of certain team members to complete their work and move the task further toward completion. Accordingly, by collecting statistics on the movement of tasks between statuses, we can see what trends are currently occurring in the team.

First, do we invest in our assessment.

Secondly, are there bottlenecks in our team.

The amount of time invested in the estimate should ideally match the amount of time the team spent developing and releasing the task. However, we can see that the task hangs in the intermediate wells.

In the example below, we can see if tasks hang in several statuses for too long. And if it is impossible to influence the timing of finding tasks in the work process, then something can and should be done with intermediate statuses such as “Ready for development”, “On Hold” or “QA Done”.

An increase in the time spent in any of these statuses tells us that the team either has very well-tuned processes, or we have problems. For example, with sprint scope planning, process problems or lack of resources.

Discuss it with the team and you’ll be surprised how much you can learn.

In any case, tracking Cycle Time will allow you to work out bottlenecks that need attention. A reduction of even a few hours in each status can accelerate the release of tasks by one or more days, therefore, more tasks can be done in the same amount of time.

Profit 🙂

Lead Time allows you to see if there are similar drawdowns at the junction of Discovery/Delivery phases. First, when raising to a level (see the pyramid above), a control vision for the work of analysts/architects appears. As an additional bonus, you can detect problems with the same statuses and loss of time in situations where the task has been ready for development for a long time, but the Delivery team does not start it. Maybe it should be like that, maybe not. However, fixing processes and eliminating downtime will also affect the volume of functionality released for the better. If this is not possible, at least pay attention to the stability of the trend.

Profit x2!

And finally, Time to market. This is a Helicopter view over all stages. Look at the stages of value delivery from above, evaluate the entire process. Even if you see that there is nothing to fix before Discovery, tracking Time to market and other metrics over a long horizon allows you to see the volatility of the team’s work. Are there jumps? Are the processes in the team stable?

Conclusions

If we talk about the initial problem, the main goal of the product team should be to set “More value with maximum quality in minimum time”. And the metrics will be our speedometer, showing how fast we are moving along our Path with this setup.

What else do these metrics give us?

At the beginning of their implementation, the collection of metrics will show you the current state of affairs. Are the deliverable scope tasks sufficiently decomposed (if you use SCRUM). Over time, if it hasn’t already happened, you’ll accumulate statistical information about the team’s competencies and the presence/absence of resource hunger. After all, the idea of ​​the team that their work is measured by numbers is a good incentive for the team to tighten internal discipline, and for those suffering from “Student syndrome” to cheer up and think about the fact that this is dangerous for their career.

Later, you will be able to move on to setting standards that will allow you to see deviations from the norms. And most importantly, raise the team to the level where it itself will help you improve process metrics and your performance. After all, a team differs from a working group in the adoption of common goals, and not in the simple achievement of results;)

Write in the comments if the article was useful.

If there is a request, I will definitely tell you in more detail how and in which tools you can calculate the metrics of production processes.