TVP – Metric 38 Delayed Stage Kills

Background | User Guide | Program Contents | Stakeholders | List of Metrics

1. Metric Definition

A measure of the failure of projects to move beyond a given stage; particularly in reference to the later stages of the product/process development activity.  The most frequently used form of this metric is the number of projects that are terminated either in commercial introduction or in the last phase of development.

This metric is central to the basic notions of R&D yield, productivity, effectiveness, and risk and performance abilities. It is also multi-functional in character.

This metric is defined as the failure of projects to move beyond a given stage; particularly in reference to the later stages of the product and process development activities. Since the purpose of R&D is to produce knowledge that reduces risk, then it follows that later stages should be represented by projects that have been risk-reduced. If this is so, there should be more successes and fewer failures at later stages. Conformance to this goal is captured by this common metric as expressed in its negative form.

The most frequently used form of this metric is the number of projects that are terminated either in commercial introduction or in the last phase of development. However, this can also be generalized by examining the actual failures, or rejections, at a given stage and comparing this to a predicted level. An example might be that at a certain stage it is expected that only ten percent of the projects should fail; instead, 30% are failing. This indicates a later stage kill ratio that is three times higher than expected. There is debate as to whether this metric should be measured only at the stage in question or should be made on a cumulative basis that integrates all future failures that occur beyond a particular stage. This may be more appropriate to some organizations. The important concept is to establish a useful measure of a result to be avoided, i.e. making decisions to terminate later than they could have been made. Overall, these metrics are critical in describing the kind of risk that can be attempted and executed well for a given innovation portfolio.

2. Advantages and Limitations

The advantages of these late-stage kill metrics is the focus they provide on the cost of ineffective processes (fail early, fail often, fail cheap). No one wants to spend time and money in significant amounts at later stages just to see it all thrown away because of an unanticipated failure. This is particularly true if the cause of the failure should have been uncovered by the R&D process at an earlier stage.

These metrics capture the degree to which R&D is fulfilling its risk reduction role for the corporation, thereby increasing its contribution to the value of the enterprise. Because of the value of the metrics in this regard, they are useful for targeting future R&D performance.

A limitation of this metric is that it only represents part of the innovation process that involves other functions besides R&D.  This metric is also a lagging indicator and therefore there may be a significant time gap between problems in the innovation process beginning and this metrics ability to diagnose them.

3. How to Use the Metric

The metrics should be tracked at least annually. Because of measurement and definition problems, a baseline of two years or more is needed before accurate judgments can be made about trends and ratio efficiencies.

The metrics should be examined carefully for consistency with business strategies and the results required vs. the investments in R&D. In situations where the metrics indicate a high late-stage kill, there will be a difficulty in taking on high risk programs. This means that this metric should provide a guide to the level of risk that can be undertaken in executing the overall business and technology strategies. One or the other must be shifted, and variations in how R&D is conducted and managed need to be examined.

4. Options and Variations

The most common option is to apply these metrics to all stages and not just the last one, but to  generalize to all stages. However, they lose the potency and impact of having to write-off R&D at a time period when this should not be the case.

5. Champions and Contacts

6. References