Fallacies of Effort Tracking

The discussions around tracking effort for work items of any kind in (new) product development seems to never cease. In this article, I try to lay out why tracking effort is likely a waste of time that does not help your product in any way.

Why do we track effort spent per work item anyway? There are two answers I hear regularly to this question:

• in order to learn and optimize our output
• in order to increase predictability

The following two sections will dig in each of these perspectives.

The Optimization Perspective

A common rationale for time tracking is the wish to get information in order to optimize the output. Let me give a bit of context to explain why effort tracking will not help for that.

A nice way of analyzing the efficiency of processes is Value Stream Mapping. A bit simplified, it maps the various steps a work item goes through and reflects average touch-time and wait-times. For example: After some refinement activities (one half-hour actual discussion during a two-hour meeting) and waiting for another two weeks in the Backlog, a work item might be loaded into a two-weeks sprint during planning. It might require a day to implement and another day for testing and documentation. After the Sprint, it will wait an average of 10 days until it gets deployed in the monthly deployment cycle.

In this exemplary excerpt of a value stream, we can identify that the item resides 31.25 working days in the process while being worked on for 2.2 days. This information allows us to calculate a metric called flow efficiency, which is the percentage of time an item is actively worked on (touch-time) as compared to the time that item resides in the process (lead-time). In our example that would be approximately 7%. This is a pretty representative number. In fact, we rarely see flow efficiency exceed 15% where it is not very consciously managed.

Additionally, let’s be aware that the meaningful figure for customers (those who pay our wages eventually) is actually the lead-time. Once decided that something is going to be done, the question “When will it be available?” is much more relevant than “How much effort is it?”.

All this implies that if we focus on optimizing the touch time, in our example we optimize about 12% of the lead time that is relevant to customers. The bad news here: it’s the 7% that are hardest to optimize. What we should do instead is to optimize our system to get rid of a big share of the 93% wait-time. To spend less time doing nothing with a work-item is typically much easier than doing what needs to be done faster.

To make things worse, tracking effort does not even give us any clue with regards to how to do things better. All it can show is that an item took more effort than expected. But frankly, engineers do know this anyway. Unless we operate in a system built around mistrust, why would we need to track numbers to prove to engineers that a task took longer than they expected it to take? I can’t think of an answer to this question.

Put short, from a perspective of optimization, tracking effort puts our focus on the wrong spot plus gives us no valuable information what to actually do differently. Instead it reinforces a culture of blaming much more than one of actual learning and continuous improvement.

The Predictability Perspective

The other argument I often hear is that we want to use the data to compare it with estimations in order to learn and improve predictability. While improving predictability is a worthy goal, tracking effort will not get you significantly ahead.

Drawing on the value stream mapping example from above, it becomes obvious that even if we rigorously apply analysis to estimated vs effective efforts, we only get information about 7% of what makes our lead-time – the one that we actually need to do predictions on. So even if I reduce error margins for this part, how much more predictable will I be over all?

Again, the better approach here is typically to reduce wait-times. Reducing wait-time also increases the predictability, but without trying to overcome the actual nature of innovative product development, which inherently volatile with regards to efforts. Another approach to reducing uncertainty with regards to the lead-time is typically to reduce batch-size. Split one task into multiple smaller ones that will show less variability.

What makes predicting effort difficult is the very nature of product development. We aim at building something new, something that hasn’t been there before. That makes it fundamentally different from other types of work like manufacturing, where we see much more repetitive tasks. As Don Reinertsen explains in “The Principles of Product Development Flow”: In product development, there is no added value without variance.

So, let’s face the fact: Taking pride in innovation power and at the same time asking for high predictability is kind of schizophrenic. And our the level of schizophrenia admittedly depends on the individual context, let’s acknowledge that basically we can have one or the other, but not both. And in the context of (new) product development, innovation needs to get priority.

Conclusion

What good is it to invest precious time in administrating effort spent on tasks, when all we can gain is a mere few percentage points improvement in predictability? Once more, Frederick W. Taylor’s ideas of scientific management prove outdated for today’s environment and the nature of knowledge work. Taking a more systemic perspective, we find that tracking effort actually tends to misguide our focus to the wrong aspects while even negatively impacting a collaborative learning culture.

So, what to do instead? Here’s a few proposals that certainly don’t make up a comprehensive list:

• Shape a culture of mutual trust that allows for real learning and continuous improvement
• Focus on lead times rather than effort. Focus on velocity (i.e. how much of our estimatied volume are we able to deliver in a defined amount of time) rather than effort, as velocity inherently puts lead-time in the focus.
• Optimize lead-times where it actually counts: Figure the granularity that reflects actual added value delivered to your customers. This is typically rather coarse grained than fine grained.
• Gain predictability and improved output through rigorously reducing waiting time.

But most certainly: Stop (or don’t start) doing what is not helpful.