TVP – Metric 28 Peer Evaluation

Background | User Guide | Program Contents | Stakeholders | List of Metrics

1. Metric Definition

A comparison of the firm’s technology in a particular area compared to the current state of the known art and particularly against competition, if known.  The comparison may be made by an external peers or an internal panel.

External peer evaluation is another method of assessing the strength of a firm’s technology. Alternately, an internal panel of top scientists could be assembled, however this group may be perceived less credible due to their stake in the outcome. The evaluation in either case would involve a comparison of the firm’s technology in a particular area compared to the current state of the known art and particularly against competition, if known.

2. Advantages and Limitations

This metric is a good diagnostic tool for future growth and progress if an unbiased panel of experts can be assembled. It can also be very credible if the panel members are credible to the major stakeholders; an external, blue-ribbon panel is suggested. This can lead to remedial action based on panel suggestions.
Limitations and disadvantages include: possible difficulty with benchmarking competitive technology; subjective and qualitative; potential confidentiality problems with outside panels; and further difficulty in assembling a trusted panel.

3. How to use the metric

Selection of the panel is extremely critical. People must be selected with credibility to the stakeholders as the predominant criteria. Technical competence in the field is essential, and the panelist must be impartial. Internal panels generally may be less believable to top management because of the stake they have in the R&D organization’s success. An external panel could include outside directors, local university faculty and department chairmen, consultants, and members of technical or trade organizations.

A numerical rating scale should be used. ( 5 = well above art or competition, 3 = equal to , and 1 =  well below)

Panelists should vote independently, and perhaps secretly, after extensive discussion with R&D participants.

4. Options

The use of internal panels probably would have little credibility outside of the technical community. However, it could be an excellent reality check within an R&D organization.

The best and most credible results might be obtained via paid, technical consultant teams. These organizations would have no incentive to benchmark low.

They could also conduct blind surveys to determine competitive positions.

5. Champions and Contacts

6. References

Section I. Metric Definitions