1. Metric Definition
A measurement of the skills and ability of the R&D staff to execute strategic programs.
a. Internal Customer Ratings
Internal customers rate the quality of the R&D staff on their ability to execute programs. Measures such as percentage of mileposts met vs. project plans, novelty of concepts, patentability of concepts, and competitive advantage of the technology are parameters that can be considered.
b. External Customer Ratings
External customers rate the quality of the R&D staff of their ability to meet customer expectations. Problem solving, novelty of approach, responsiveness, and knowledge of customer's operations are parameters that can be considered.
c. External Recognition
Publications in peer-reviewed or industry trade journals, external presentations, citations in the literature, invited lectures and patents are parameters to be considered.
2. Advantages and Limitations
The internal and external customer ratings measure the ability of R&D to meet customer expectations and contribute to the growth of the corporation or enhance competitiveness. They are largely objective measures that can be tied to tangible value. The value of external recognition via patents, publications, and presentations is more difficult to measure objectively. The numbers of different subjects covered by public disclosures should be evaluated rather than the total number of all disclosures. It is easy to become subverted to a self servicing metric if only numbers are considered. Maintenance of technology as trade secrets must be considered in this evaluation.
3. How to use the Metric
Internal customer surveys can be conducted using a rating scale of one to four. Level one represents below standard execution on a given project. The causes for this poor performance have to be determined since they could arise from inadequate skills, poor judgment, lack of responsiveness, and/or poor planning. The causes may not be related to the quality of the personnel but poor management practices. Having multiple internal customers (marketing, manufacturing, and sales), conduct the evaluation is a form of 360o review. Superior performance by reaching targets ahead of schedule, lower than expected costs, and/or developing a significant competitive advantage should be rated as a four. In establishing this survey system, agreement should be reached on the different levels of performance. The survey should be applied to different projects with the same population of the internal customers as raters. This rating should be conducted on a regular basis and over time trends will emerge.
External customer surveys should be conducted using the same one to four scale. Level one would indicate that the customer was not satisfied with the parameter being measured, while level 4 would indicate that expectations were exceeded. Key parameters should be selected beforehand and could include timeliness of response, knowledge of products, knowledge of customers’ operations, and knowledge of customer needs. The parameters will vary by industry. Several levels of the customer’s organization should be sampled such as plant operators, first line supervisors, and management. A simple postcard type of survey instrument mailed after customer contact can be used. A database can be developed over time and trends will emerge.
The subjects covered in public disclosure should be tabulated and compared to strategic technology goals. A subjective one to four rating system can be created by R&D management to determine fit with the goals. A rating of one corresponds to a poor fit and a need to enhance skills, while a four implies that all areas are being addressed.
4. Options and Variation
Service and consumer product companies may find the external customer survey to be a valuable tool in assessing the effectiveness of their R&D organization. One may also measure the quality by the number and type of external awards from recognized organizations (ACS, AIChE, and IRI)
5. Champions and Contacts