Hindsight is 20/12

6 08 2010

Time estimations are hard! Even Douglas Hofstadter says so:

Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.

In fact where time estimations are involved even hindsight isn’t 20/20.

Part of R&D’s responsibilities is to are the last line of defence support, if a problem stumps Support they will escalate it to R&D. However support is not considered to be a developer’s main goal in life and it’s useful to know how much time developers spend on support so as to be able to plan ahead.

This feature should take 6 man-month to develop, we have 3 developers but they spend 10% of their time on support so it will take 9 weeks rather than 8.

But how can we know how much time is spent on support?

When we rolled out our escalation management system we added a step at the end of the process in which the developer estimates how much time was actually spent on this case. We soon saw that the numbers appear to be very low for several reasons.

  1. Escalations typically have several iterations of getting more information from the customer and this may span weeks (or months if the customer has more important things to do) making it very hard to remember how much time was actually spent on the escalation.
  2. If an escalation is re-assigned the time spent by the first (N-1) developers is lost.
  3. In some cases the escalation was closed before the developer had a chance to fill in the data.

In order to combat all these problems we made the time spent on escalations accumulative, every time a developer moved the ball into the supports’  court (asking a question, sending a DLL) a mandatory field asked how much time was spent on this iteration.

Suddenly the estimated time spent on escalations went up, it would appear that when trying to recall how much time was spent doing support R&D tended to underestimate by about 40% (the actual number I got was 42 but that’s just too good to be true).

These data [1] are problematic since they contain the unrelated set of escalations that were closed before R&D filled in the time spent and the lost time of developers other than the last one. Putting that aside what we can see from the graph is that retrospective estimations tend to underestimate time that was spent treating an escalation. The peak at zero[2] is probably to do with the causes mentioned above but we still see that on average the estimations are lower than those gathered when incidents were fresh in memory.

This is just one data set, I would be interested to know if there are other such data available.

Maybe this has something to do with the notorious difficulty people have in estimating how much time something will take, if you can’t remember how long a similar task took how can you be expected to learn from past experience? Perhaps if we kept better track of our time we would slowly learn how to make better estimations, even those about the future[3].

[1] I’ve omitted the long tail of escalations that took more than 8 days since they add little to the general picture.

[2] Zero days is not obviously an error, time is measured in hours and then converted to work days. Therefore any amount of time up to 4 hours will be shown as 0 days. Even when accumulating the time we see that the highest proportion of escalations were solved in 0 days although this slice is half as big as any other.

[3] As Niels Bohr said Prediction is very difficult, especially if it’s about the future.




2 responses

7 08 2010

nice reading. In my opinion there are several major obstacles to successful estimations of time required to handle for future support cases which make “better time tracking” useless:
1 – statistics fail to predict how many prioritized escalations would appear
2 – statistics fail to predict how responsive customer would be and how effective would be communication iterations with other support levels
3 – support cases are simply different each time. Similar cases shouldn’t ever reach Tier3 (in ideal world of course).
Thus, estimations are typically based on weak assumptions (even if developer keeps precise track of time).
It would be interesting to hear manager’s feedback to this post.

8 08 2010
Motti Lanzkron

I agree that predicting time spent on support is problematic for all the reasons you mention. In my last point about time estimates, I wasn’t referring specifically to support. I think that we can extrapolate the fact that people mis-estimate the past when dealing with support to other tasks, if this is true in general it may reflect on the reasons we have trouble predicting how long a given task will take.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: