Community Forum – Measuring R&D Project Performance

How do you measure performance of R&D projects?

We are considering implementing a model to measure/evaluate performance of R&D projects, specifically, their EFFICIENCY. (EFFICIENCY = OUTPUT / INPUT) When we finish a project, we can have various outputs (IP rights, papers, etc.), but it is not easy to evaluate performance with those outputs from a financial point of view. Our questions are as follows:

1. What methods do you use to evaluate performance (or efficiency) of R&D projects?
2. What metrics(index or indicators) do you use to measure performance (or efficiency) of R&D projects?
3. For the Service Industry, it is difficult to evaluate performance of R&D projects because their outputs are IPR, developed services, etc. and often don’t have financial values. With these outputs, how can we evaluate performance or efficiency?
4. When you evaluate R&D projects, do you measure performance by FINANCIAL VALUE? Thank you.

– R&D Planning Team, Technology Strategy Department, Global Information & Tele-Communication Company

Community Responses

Response 1
Project Evaluation:

It’s one thing to measure the intrinsic business and profit potential of an opportunity for investment and another to quantify the “Fit Factors” that will be necessary to develop, scale-up and commercialize that opportunity. You might want to check the article: “Constraint Analysis for Assessment of Business Risks.” J. Technology Management  Vol I, No. 2. 1994, pp 42-53. Statistically, it has produced 8 or 9 out of 10 commercial successes for early-stage discoveries. But it is equally useful for evaluating your current businesses, (actually a bit unnerving and a good place to start)…Critical factors in evaluating the intrinsic business attractiveness include estimates of potential sales and profits (including for each application of the technology), if it’s a “platform” technology, Identification of the easiest route to first market entry is important; Quantification of the growth potential for each segment is needed (is this a rising or declining tide); How strong is the proprietary protection, and the strength of fast-follower reaction here and abroad; What is the degree of its disruptive character that might restructure an entire industry including the ones in which you now are involved; Are there significant political/social incentives or dis-incentives?.

In-house availability of all the Fit factors used to be a knock-out factor, but the creation of the Limited Liabilty Corporation (LLC) model of organization and management, means that any operating factor: manufacturing, marketing, distribution, specialized technical resources etc. no longer have to be in-house, but can be provided by limited partners. Special attention must be paid in this (“outsourcing world” to multiple secure-sources of materials and components. Regression analysis has provided the weighting for each factor.

The system works for any kind of business, including services and the initial scoring can be done on the back of an envelope to see if you are in the ballpark, and where detailed analysis may be needed.

Hope this is helpful…

– Bruce Merrifield, IRI Emeritus

Response 2
One way to evaluate  project efficiency will be to measure the total project cost and evaluate it against the revenue, or preferably, the profit (EBIT) it contributes to the business.  Profit contribution comes from either bottom line cost reduction or by top line business growth (profit add).  A good project will launch and validate itself by a payback period in 3-5 years depending on the scope of the work and magnitude of the project.  Obviously, a shorter time period is better; but growth curves for new product launches can get extended.  In that case, a magnitude multiplier (profit/investment) over a set time period will work. Cost reduction projects typically have shorter payback periods <3 years.  In either case, a rate of return and discounted payback can be used to measure the relative success of the effort.

– John Rovison, FMC Corporation

Response 3
For development, the industry focus is on business return. This is understood. But for research, it also has some academic meaning, such as publication and impact on some area later. This may not be evaluated by business return in the short term. Therefore, R&D projects should have two-fold impacts, academic and business, in my opinion. We cannot only forcus on one side. Thanks.

– Zhang Xiaomin, Senior Scientist II, BASF

Have a response to add?  Email us!