INNOVATION PORTFOLIO ARCHITECTURE

An innovation portfolio is designed to funnel ideas from initial conception to product development. Unlike a project portfolio, which is focused on managing products in development and guided by a clearly defined strategy, an innovation portfolio is a collection of concepts loosely organized around an emerging strategy.

Scott Mathews

Scott Mathews is a Boeing Technical Fellow and technical lead for the Business Engineering initiative within the chief engineer's office of the Boeing research and development division. He provides technical consulting to business units for developing investment and risk models for new products, strategically significant projects, and innovation portfolio management. He is an internationally recognized expert in complex financial and investment decision modeling that features real option valuation. For the past 20 years, he has been engaged in stochastic modeling, capital markets investment, and financial and strategic analysis. Previously, Scott was employed in the United States, Europe, and Asia as a design engineer in robotic control systems, artificial intelligence, and systems and software development. He has academic degrees and training in computer engineering, digital control systems, artificial intelligence computer science, and computational finance. Scott chaired the session on Portfolio Management at the 2010 IRI Annual Meeting. scott.h.mathews@boeing.com

OVERVIEW: An innovation portfolio, which is focused on early-stage ideas whose role in the overall strategy is still evolving, is very different from a project portfolio, which is focused on managing products in development. Our company sought to implement an innovation portfolio program to manage the development of concepts from initial idea to the front end of the project portfolio. The design of the innovation portfolio was motivated by the desire to increase the number and quality of projects entering into the project portfolio process. The innovation portfolio must allow for an evolving strategy as the most promising concepts emerge, leveraged by incremental investments. Unlike a project portfolio, attrition in the innovation portfolio will be very high, with concepts being shelved at each phase of analysis. The innovation portfolio is still in the process of thorough evaluation, but initial results are encouraging.

KEY CONCEPTS: Innovation portfolio, Strategic portfolio, Portfolio management, Valuation, Real options, Datar-Mathews Method, Strategic decisions.

We clearly had a challenge in front of us. Our traditional markets were undergoing a radical restructuring and downsizing. Our customers were demanding substantial changes to accommodate their new realities in the field. And competitors were emerging from unexpected niches within the industry. We needed both a way to capture more internal innovative ideas and a system for managing and funneling these ideas into existing product-development processes.

The objective emerged after discussions with senior management: develop a system to rapidly examine 100 new ideas for products and services each calendar quarter, select the 10 best, and develop them into product prototypes for market trials. This innovation pipeline, which would deliver more and better concepts into our development process, was how we would get out in front of our competitors and put new, innovative products into the hands of our customers more quickly.

The company already had a well-practiced approach for developing a product once it passed the initial review at gate A, the point at which every product concept was matched to a market customer, aligned with strategy, and assigned a development team with a budget. Substantial resources were frequently committed to projects at gate A or shortly thereafter. Subsequent gates included the usual checks on execution of project deliverables, specifically budget, performance, and schedule. Often, several projects were bundled together in what was termed a “project portfolio.” These project portfolios didn’t really operate much like those described in the literature on the subject (see, for example, Cooper, Edgett, and Kleinschmidt 2002a). Project portfolio managers had few of the prerogatives usually reserved for portfolio managers, such as establishing the strategic alignment, determining the right number of concepts, or balancing the selection of concepts. Instead, the focus was on maximizing the value of the existing portfolio through tactical management of resources. Attrition rates were typically fairly low—in the neighborhood of 20%—partly because the projects were deemed essential to an established strategy, but also because of the resources that had already been committed. With such restrictive practices common to many of our project portfolios, we quickly realized that a project portfolio would not deliver the capabilities senior managers were seeking.

Figure 1.—The innovation portfolio connects ideation to product development.

Clearly, a different portfolio management process was required to bridge the gap between ideas and development-ready products—a strategic or innovation portfolio model. An innovation portfolio has a different objective than a project portfolio and differs in significant ways (Table 1). Where the project portfolio is focused on execution and delivery, the innovation portfolio concerns itself with the development of a coherent portfolio strategy and the maturation and selection of project candidates, called concepts. The innovation portfolio, then, connects existing ideation events, where ideas are born, and project portfolios, where matured concepts are developed into products and services (Figure 1; see also Cooper and Edgett 2009).

Innovation Portfolio Design Principles

Designing an innovation portfolio requires a foundational understanding of the behavior of the innovation process and how it differs from the new-product development process (see Terwiesch and Ulrich 2008; Rosenø 2008). A project portfolio, which manages projects through a quasi-deterministic process with low attrition rates, differs from an innovation portfolio, which is managed through a complex and emergent process, in part because of the high uncertainty and resultant high attrition rates of the concepts themselves. Unlike a project portfolio, which is governed by a closely defined strategy (Cooper, Edgett, and Kleinschmidt 2000), an innovation portfolio is typically initiated with only a weakly defined strategy, perhaps more accurately called a “strategic intent.” The strategy details emerge as the most promising concepts surface and uncertainties are resolved. The complex interaction between the developing strategy and the nonlinear value trajectory of the concepts means that change in one aspect of the innovation portfolio produces a confounding effect on other areas. This in turn impacts the whole innovation portfolio.

The complexity of the system results from the impact of the emergent strategy. Optimizing the behavior of the whole innovation portfolio may require suboptimal behavior in some portions. Simply selecting the best concepts according to some defined criteria, say estimated value, may give the illusion of superiority, but executing that strategy ultimately may prove discordant. Instead, an innovation portfolio includes the best set of concepts that support a coherent strategy, with the awareness that while a few concepts may not provide the same returns as others, the overall aggregation has a high value-creating potential.

The concepts themselves are subject to evolutionary pressures within the portfolio, with only those demonstrating positive value momentum and resolution of risk challenges surviving to the next incremental investment phase. In making a decision to shelve a particular concept, the manager can be secure in the knowledge that the portfolio will always include several competing concepts that better support the emergent strategy.

The intended deliverable of an innovation portfolio, then, is a well-articulated strategy aligned with a handful of promising concepts that make the strategy feasible (Say, Fusfeld, and Parish 2003). These concepts are released into the project portfolio process for substantial funding, resource management, and execution. In terms of the portfolio control system architecture, the innovation portfolio is a positive feed-forward, open-loop system for evolution and discovery; the project portfolio is a negative-feedback, closed-loop system for delivery.

The conundrum of an innovation portfolio is that while it encompasses important strategic issues, the individual concepts are often not ready for presentation to senior management. The reason is that the early-stage concepts are highly malleable as a result of constantly evolving objectives and the uncertainty of value determinants; therefore, they merit only tiny investments relative to the large funding needed for full-fledged projects. Furthermore, these immature concepts often entail technologies or customer specifications that may be difficult to grasp because they reside in adjacent or far-adjacent markets. In this context, it is impractical to require senior management to devote time and attention to understanding and approving decisions regarding individual innovation concepts.

To solve this dilemma, innovation portfolio management encompasses a process that is tactical in nature but achieves strategic objectives. The key is for the innovation portfolio analysts to group or “cluster” concepts around strategic thrusts. Various details of the strategy (e.g., long- or short-term launch, light or heavy investments, larger margins or substantial revenues) remain to be teased out. Clusters of concepts within the portfolio will emphasize one or more of these strategic aspects. Furthermore, the clusters can be comprised of concepts of differing phase maturity, which then provide the portfolio manager a complete overview of all concepts within the portfolio. The result of the clustering is that, at periodic reviews, senior managers’ time and attention can be focused on deliberation around a set of clearly differentiated strategic choices and the near-term tactical decision on the aggregate (and therefore substantial) incremental investment requirements for the portfolio, rather than the particulars of the early-stage concepts themselves.

A coherent and comprehensive strategy will emerge over time as the portfolio manager successively selects for incremental investments one or another of the clusters for further incremental investment, maturing the underlying concepts, while setting others aside. With this process design, the strategic innovation portfolio operation takes on the rhythm of tactical execution, with scheduled quarterly reviews, near-term investment decisions, and a targeted deliverable: a set of matured concepts and a carefully scrutinized strategy for the project portfolio.

System Design Challenges for an Innovation Portfolio Architecture

The innovation portfolio architecture must make a complex process intuitive and transparent. In turn, the well-designed portfolio provides a corporate R&D perspective that aids senior management as they align investment decisions with mature and feasible strategies for new products and markets.

To achieve this goal, the design of an innovation portfolio architecture must overcome several conceptual challenges:

• Concepts have little value, at least initially.
• Most concepts have available only a small amount of “working” data, at least initially.
• Most concepts aren’t worth pursuing very far, but all must be developed to some point in order to identify worthy concepts.
• The worth of a particular concept is difficult to determine without a well-articulated strategy (which does not yet exist).
• The wide mix of concepts makes comparison challenging.

Portfolio Management Software Systems

A survey of the portfolio system industry finds that most offerings are intended to support project portfolios; some of these are fairly large, expensive enterprise-scale systems for tracking hundreds of projects. These systems serve as an interface to thousands of individuals for resource tracking, development scheduling, and project delivery. A few companies offer strategic valuation portfolio systems that consist of a database, an analytic engine, and graphics and reporting capabilities for executive and administrative control. These typically are small- to medium-sized systems, as they focus on strategic decisions and their usage is in the hands of a relatively small number of managers and analysts.

One of the several suppliers of strategic valuation systems is Enrich Consulting (http://www.enrichconsulting.com). Enrich has had extensive engagement with pharmaceutical companies; the pharmaceutical industry has been using strategic portfolio systems for nearly a decade, and as a result, pharmaceutical companies tend to be sophisticated in their approach to these systems.

The Enrich system encompasses a database, an analytic engine, and a web-based enterprise interface and reporting system. Enrich also offers an application layer for valuation, but customers can design their own to accommodate specific corporate needs. The application layer typically consists of:

a) The interface for the front and back end of the system to fit with existing company systems in the company,

b) The analytics and process to evaluate the concepts, and

c) The user interface.

Enrich provides extensive guidance in the design and installation of strategic valuation portfolio systems.

The challenges can be addressed by careful attention to the architecture of the system. The low value and minimal initial dataset associated with most concepts can be addressed by keeping early-phase evaluations cheap and fast to minimize expenditures on very immature concepts and by keeping initial entrance thresholds very low. The earliest phases of portfolio evaluation involve qualitative and rough order of magnitude (ROM) estimates. The difficulty of aligning concepts with a strategy that is still emergent can be ameliorated by clustering early-stage concepts around particular strategic thrusts; as a strategy emerges, those clusters not aligned with it can be eliminated. And, as the strategy matures, concept clusters can be scrutinized with increasing detail. And finally, evaluation of diverse concepts can be eased by a system that includes the minimum number of broad attributes necessary to effectively evaluate concepts.

Each of these architectural decisions drives a concept maturation process that allows the innovation portfolio to evolve a collection of loosely connected, very early stage ideas to a cluster, or set, of matured concepts that enter into the project portfolio process with a clear driving strategy. The process that we have developed to address these issues begins with interface to the ideation system, proceeds through an initial coarse screening and several phases of concept maturation, and ends with the integration of remaining concepts into the project portfolio.

The Front End: Ideation

While the ideation system must feed into the innovation portfolio, the ideation process should not be a structural part of the innovation portfolio system. An ideation system has significantly different requirements that necessitate a different architecture. Although the innovation portfolio obviously must accommodate the usual flow of idea inputs from individuals and managers, in most companies, ideation events provide by far the largest source of new ideas flowing into the portfolio. At its most fundamental level, a good ideation event is a social networking process engaging broad swaths of the corporation. For example, our ideation events have attracted upwards of 1,000 individuals (managers and engineers) and produced hundreds of new, valuable ideas targeted at differing strategic opportunities. In contrast, a portfolio is an analytical database and typically the exclusive purview of management for investment decision making. Clearly, these are two very different functions requiring different management approaches, which should be reflected in the system architecture.

In our case, the company already had a well-functioning web-based ideation process, and the innovation portfolio system was designed to mesh seamlessly with that existing structure. A typical ideation event will generate hundreds of what we call “idea fragments.” An idea fragment is one element of several needed to qualify a concept as a well-defined opportunity with market value. A proposal to develop a promising technology, something we see plenty of in an engineering company, is an idea fragment because it lacks, among other things, a tangible product suggestion, a means to manufacture it, or an identified market customer. Idea fragments by themselves are insufficient for analysis within a portfolio. Therefore, as part of a cataloging and indexing process following ideation events, we cluster idea fragments together in patterns that complement their development and that ultimately may connote market value. These clusters of idea fragments then form the kernel of what we term a concept.

The threshold for entry into an innovation portfolio should be as low as possible to minimize premature rejection of potentially successful concepts, called Type 2 errors. A fast and frugal decision tree (FFDT), a heuristic tool emerging from psychology (Gigerenzer and Goldstein 1996; 2002), is perhaps the most efficient and easiest approach to setting the minimum criteria for entry into the portfolio to its lowest analysis level (Figure 2). FFDTs are simple enough to operate effectively when time, knowledge, and resources are limited—such as when several hundred concepts must be evaluated quickly. The FFDT does not assess the concept's potential value or likelihood of success. Rather, it sets a minimum level of quality that suggests a concept merits further attention within the context of the portfolio. Using this initial screening process, as many as one hundred concepts (clusters of idea fragments at this stage) can be ready to enter the portfolio soon after an ideation event.

Phase 0: Coarse Screening

Concepts enter into the portfolio at a Phase 0 level of assessment, the qualitative coarse screening. Coarse screening is similar to how we set boundaries on a personal dating pool. It defines a minimum threshold for those people we are willing to consider for at least one date—six feet tall, athletic, doesn’t smoke, and, of course, likes those long walks on the beach. In the portfolio process, coarse screening allows rapid but relatively shallow evaluation to separate out those concepts that we are willing to spend more time with.

Figure 2.—A fast and frugal decision tree (FFDT) can provide an efficient mechanism for setting criteria for entry into the innovation portfolio.

In the innovation portfolio process, qualitative assessments form the basis for the coarse screening of concepts to determine whether they attain the minimum standard for value and success likelihood (MacMillan and McGrath 2002; Terwiesch and Ulrich 2008). Our coarse screening is conducted with a half-dozen questions that are scored, occasionally weighted, and then summed and totaled to yield a ranking. The IRI anchor scales (Scriven 2001) are a good initial set of coarse screening questions. The questions are related to the overall value fit and success likelihood within the organization, including such issues as resource and organization alignment, competitive advantage, and IP position. At this point, strategic alignment should not be considered. Since innovation portfolio strategy is an emergent phenomenon, concepts will drift in and out of alignment as the strategy is refined and the development team learns more about the concept itself. Not considering alignment also avoids the conundrum that arises when a concept is deemed to have high value, but low alignment to strategy (or vice versa). Does that imply someone has erred in specifying the strategy? Admittedly, such an oxymoronic situation can occur in some dating situations, but you don’t want your portfolio burdened with the same type of contradiction.

Phases 1–3: Maturing the Concepts

The innovation portfolio captures the concepts’ provisional progression through the portfolio via a phased structure (Table 2). Architecturally, the user interface for each of the phases is designed to elicit progressively more detail about a concept. The intent is to gather just the necessary and sufficient information to justify allocating the minimum funds to advance a concept to the next phase or relegating it to the repository. Thus the phases are increasingly more costly, in the sense of the level of effort required of the analyst and technologist. Phase progression, or maturation, is approximately correlated to the level of confidence in the fidelity of the value estimates of the concepts, though not necessarily to the level of uncertainty of the value estimates. As an example, one can be confident about an estimate that has been thoroughly researched and documented (Phase 3), but that estimate can span a fairly wide range, such as an assumption about the range of unit sales that is dependent on market scenarios.

As a concept progresses through the phases, it is in a state of constant evolution, to the point where forecasts about its future state are highly unpredictable. Many concepts are in a state of high optionality, meaning that their value and success prospects are contingent on the outcome of a plethora of assumptions, most of which are not under the control of the concept team. Concepts unable to resolve uncertainties around these assumptions or advance through the phases are shelved in favor of those that can. The portfolio is indifferent as to how the progress is achieved—whether by research, expert opinion, and Delphi surveys; laboratory experiments; construction of small prototypes; an external study; or the services of a consultant.

Phases are different from the “project gates” that show up in so much project management literature. While stop/go project gates often yield excellent results by allowing a manager to check off all the previously agreed-to project milestones before he writes the next check (Cooper, Edgett, and Kleinschmidt 2002b), such gate reviews can be disastrous when applied to innovation portfolios, because the concept itself is subject to being substantially morphed by merging with another concept, partitioned into divergent concepts, or shelved altogether. Staged gate reviews rapidly become unmanageable in the context of an innovation portfolio that includes small investments in scores of constantly evolving concepts. On the other hand, the phase approach helps ensure efficient use of the analyst's time, allowing quick decisions; the analyst cannot devote too much time to any one concept, since many will eventually be shelved. Unlike the linear progression of the stage-gate process in project portfolios, concept detail can be added within the innovation portfolio at any time in any order.

As detail is added to each attribute, the analysts’ confidence in the information gathered increases. Although they differ in level of detail at each phase, the valuation calculations for all phases remain comparable, enabling the portfolio manager to make informed investment choices between more-or-less mature concepts. Furthermore, the portfolio manager is free to view, cluster, compare, and select equally from all concepts regardless of their current phases; however, recognize that the fidelity of the estimates may differ among the concepts of different phases. This operational factor of the phase approach contributes tremendous flexibility to the evaluation process, neatly accommodating and organizing the somewhat chaotic nature of discovery in early-stage concepts.

Figure 3.—Quantitative assessments in Phase 1 are captured via rough order of magnitude (ROM) estimates arrayed on an exponential scale.

Quantitative attributes, which may be elusive in the first stages of analysis, can be defined with increasing precision and fidelity as the concept receives incremental attention. Phase 1 emphasis is on understanding the business value proposition, both contextually and quantitatively. The quantitative threshold at this point is quite low; the analyst selects a “rough order of magnitude” (ROM) estimate, essentially a bucket value representing the most-likely scenario, for each of the attributes (Figure 3). The selectable values are arrayed along an exponential scale that span the possible range of the concepts held in the portfolio. This very visual and simple tool deftly bridges the apparent chasm between the qualitative Phase 0 approach and quantitative Phase 1 analysis, without overburdening the analyst at this early stage. The bounds of the “bucket” are values mapped to a numerical scale associated with the qualitative NASA Technology Readiness Levels (TRLs; Mankin 1995).1 Since technologists can readily assess the TRL of a technology project, but not as easily the quantitative uncertainty bounds of a cost estimate, this mapping provides another rapid, inexpensive method of arriving at an initial concept valuation. On average a concept requires less than a day of analyst time to assemble the requisite quantitative estimates and document the underlying assumptions, as well as the associated market and technology data that form the basis for the business value proposition.

Once attribute values are assigned to all concepts, then the portfolio process can graphically display the relative comparison of the aggregated values of clusters of concepts arrayed amongst differing strategic thrusts. The portfolio manager selects those clusters, or strategic thrusts, most in tune with corporate capabilities and direction. This process thus shelves many concepts with less merit, effectively reducing the number of viable Phase 1 concepts by half, to approximately 40. The remaining concepts are promoted to Phase 2 with concomitant incremental investments.

In Phase 2, the focus is on a better understanding of the risks and opportunities that are implicit in the assumptions. The analyst adds more detail to the initial ROM estimate by specifying optimistic and pessimistic scenario values, resulting in what we call a “range value” (Figure 4; see Mathews 2009b). The scenarios are designed to elicit the range of outcomes that could emerge from variances in the underlying assumptions, providing key insights into major risks and opportunities that might arise later in the project's tracking. The default range values are those assigned by TRL reference in Phase 1, but they can be modified by the analyst in Phase 2 if a better estimate becomes available. Technologists easily identify with the range values as a proxy for confidence ranges, with which they are familiar from engineering studies. Phase 2 concepts each require perhaps a couple of days of analysis. Once again, the attribute values of each concept become the aggregated values of clusters of concepts. As mentioned above, clusters are comprised of concepts of differing phase maturity (Phase 1, 2, and 3). The progressive selection of clusters effectively reduces the number of viable Phase 2 concepts by half, to approximately 20. These are promoted to Phase 3 analysis.

Phase 3 is concentrated on deriving the elements of the business case and the particulars of the investment requirements. The Phase 3 analysis develops cash flows for each of the scenarios attached to each of the remaining concepts; this process requires two to three days of analysis time for each concept. Special attention is given to describing how a portion of the initial development investments will be targeted for risk mitigation. At this point, sufficient information has been gathered to assemble the elements of a business case and provide the other information required for the concept to be promoted with confidence to gate A—the initial evaluation for entry into the project portfolio—and into development. At this stage, as few as five out of the initial 80 concepts may be chosen for promotion to the project portfolio.

Figure 4.—Phase 1 range values (uncertainty bounds) supplement most-likely ROM estimates based on a quantitative scale associated with NASA Technology Readiness Levels (TRL). In Phase 2, the analyst can override these range values if better estimates are available.

The Back End: The Project Portfolio

The objective of the innovation portfolio is to deliver more, higher-quality concepts into the first gate (Gate A) of the project portfolio process. The concepts that survive the innovation portfolio process have been found to be on the right side of increasingly refined strategic selections. These few concepts will have been advanced through the process, spurred on by incremental investments that pay for ever-closer scrutiny.

By the end of the innovation portfolio evaluation process, most of the elements that comprise a business case will have been assembled and documented for the surviving concepts, providing relatively detailed plans for near-term preproduction development and the accompanying investment needs, as well as an assessment of the uncertainties or risks that remain to be mitigated. All three elements are required to pass gate A, the initial review for entry into the project portfolio.

Meanwhile, scores, perhaps hundreds, of concepts will have been examined at varying levels of detail within the innovation portfolio, and the large majority, perhaps as many as 90 percent, set aside in a repository. These shelved concepts are saved with their respective descriptions and analyses to be revived if, or when, conditions change sufficiently that they merit reexamination.

The Innovation Portfolio Evaluation: Selecting Appropriate Attributes

The selection of the right set of attributes for concept evaluation is at the heart of the portfolio architecture. The attributes must answer the most relevant questions a manager can pose about the prospects of a concept, yet not be burdensome for analysts who must examine many concepts. Increasing the efficiency of the portfolio process means gathering information on the minimum number of elements needed to capture the essential decision dynamics required by the manager. We decided on just six quantitative attributes (Table 3). Ultimately, a portfolio must enable the comparison of concepts. Just these few attributes can encompass the many factors that characterize investment potential in risky concepts. By assembling the attributes in the right combinations, the portfolio evaluation is able to answer questions such as “What is the value of any one concept?” and “Why would a manager select one concept over another?”

The attributes we selected have strong parallels to the factors that comprise option valuation; option techniques are the sine qua non of concise investment analytics for uncertain propositions (Mathews 2009a; Mathews, Datar, and Johnson 2007). The value outputs are calculated from a combination of the various attributes. With adept use of discounting methods, the portfolio is able to calculate a variety of output value metrics borrowed from net present value, option, or nondiscounted valuation methods. A future article will cover the details of how this is accomplished.

Because the valuation outputs are calculated on the same basis for all concepts, the outputs can be amalgamated across groupings of concepts which emphasize one or another strategic thrust. Additionally these universal outputs allow tracking of value momentum as the concepts transit through the different phases and specific risks are addressed and opportunities confirmed. This capability then enables the manager to make better, more informed selections about strategy and the underlying concepts in the context of an innovation portfolio.

Portfolio outputs

Value metric outputs are most useful when they are compared across concepts. Our portfolio system displays output valuations for dozens, even hundreds, of concepts in well-organized graphics, such as bubble charts, cash flow forecasts, and heat maps. The system can display not only individual concepts, but also groupings or portfolios of concepts. For instance, in preparation for a periodic portfolio review, an analyst can group concepts according to their ability to contribute to a particular strategic thrust. Concepts can be grouped according to whether they have a short- or long-term impact in a market place, by level of risk, or by the CEO's favorite choices. Differences among the portfolios are delineated by graphical displays of the amalgamated output valuations (Figure 5).

With this tool, the portfolio manager is able to choose which portfolio of concepts is most appealing based on the graphical analyses, his risk tolerance, and personal intuition. In deciding to invest in one group over another, the manager adds definition to the portfolio strategy, while simultaneously narrowing the set of concepts that will eventually be advanced to the project portfolio.

Launching the Innovation Portfolio

The launch of the innovation portfolio presents challenges of its own. The innovation portfolio system differs substantially from how most pre-Gate A investment decisions have been made in the past. In our company, the early phases were quite loose. Often, a manager, or a small group of managers and technical personnel, would examine a list of potential concepts and determine which ones merited investment. Sometimes, the selection was fitted to a predetermined strategy. Although on the surface this process appears expedient, it is difficult to argue that it is thorough and methodical, or even that outcomes are any better than would be achieved by chance.

Figure 5.—Portfolio management tools allow analysts to graph groups of related concepts, as well as individual concepts. Here, the utility is used to compare various portfolio scenarios.

As the innovation portfolio was entirely new, the company decided to experiment with the process within one of the advanced development groups, prior to launching throughout the company. This group was charged with determining where to invest in a small set of proposals for prototypes chosen from dozens of possibilities. The organization also had a limited budget, which forced the kind of constraint-based decision making necessary for good portfolio management.

Preliminary results of this initial deployment have been encouraging. The innovation portfolio tool provides clear strategic choices to senior management, with the consequences of particular choices presented within a carefully articulated analytical framework. The subdivision finance and management professionals have embraced the analytic assessments and are now in the process of sorting portfolio concepts among a half-dozen different strategic thrusts, ranging from limiting investment funds to maximizing return on investment. I anticipate that the portfolio will continue to be used in ongoing decision making within that organization. Additionally, the parent organization is sufficiently intrigued that it is now examining the innovation portfolio system to replace the complex spreadsheet-based system it now uses to gather and record data on new concepts under consideration.

Conclusion

The innovation portfolio serves as an important evaluation and alignment system between ideation events and project portfolios. It is a complex, adaptive system; because of the high uncertainty of the early-stage concepts, portfolio strategy itself emerges only as the most promising concepts surface and uncertainties are resolved. Because of the high attrition rate of the concepts, a principal objective of the innovation portfolio architecture is to reduce the cost of the analysis through the application of a minimum set of attributes, based on the principals of option pricing. Furthermore, the use of phases delivers significant flexibility to the evaluation process, allowing analysts to calibrate their level of confidence as information is gathered and organized for each concept. The deliverable from the innovation portfolio is not only more and better-evaluated concepts, but also a thoroughly examined strategy aligned to those concepts. This best set of concepts supported by a coherent strategy is able to deliver more value-creating potential to the project portfolio and the corporation.

Beyond systematizing the maturation of early-stage concepts, the innovation portfolio supports innovation in other important ways. It can pave the way for the broad incorporation of innovation into the organizational structure called for by previous IRI work (See, for example, O’Connor and Ayers 2005; DeMartino and O’Connor 2006; Paulson, O’Connor, and Robeson 2007; Robeson and O’Connor 2007). The innovation portfolio architecture provides a tool that can be broadly embraced and incorporated into the organization.

¹ Boeing has derived quantification values associated with the qualitative technology readiness levels (TRLs; Patent No. 7627494). The values are used as an initial estimate of uncertainty for the cost of a technology project.

References

Cooper, R. G., and Edgett, S. J. 2009. Generating Breakthrough New Product Ideas: Feeding the Innovation Funnel. Product Development Institute.

Cooper, R. G., Edgett, S. J., and Kleinschmidt, E. J. 2000. New problems, new solutions: Making portfolio management more effective. Research-Technology Management 43(2): 18–33.

Cooper, R. G., Edgett, S. J., and Kleinschmidt, E. J. 2002a. Optimizing the stage-gate process: What best-practice companies do. Research-Technology Management 45(5): 21–27.

Cooper, R. G., Edgett, S. J., and Kleinschmidt, E. J. 2002b. Portfolio Management For New Products, 2nd ed. New York: Basic Books.

DeMartino, R., and O’Connor, G. C. 2006. Organizing for radical innovation: An exploratory study of the structural aspects of RI management systems in large established firms. Journal of Product Innovation Management 23:475–497.

Gigerenzer, G., and Goldstein, D. G. 1996. Reasoning the fast and frugal way: Models of bounded rationality. Psychological Review 103(4): 650–669.

Gigerenzer, G., and Goldstein, D. G. 2002. Models of ecological rationality: The recognition heuristic. Psychological Review 109(1): 75–90.

MacMillan, I. C., and McGrath, R. G. 2002. Crafting R&D project portfolios. Research-Technology Management 45(5): 48–59.

Mankin, J. C. 1995. Technology Readiness Levels: A White Paper. Advanced Concepts Office, Office of Space Access and Technology, NASA. http://www.hq.nasa.gov/office/codeq/trl/trl.pdf (accessed August 26, 2010).

Mathews, S. 2009a. Valuing high-risk high-return technology projects using real options. In The Handbook of Technology Management: Core Concepts, Financial Tools and Techniques, Operations and Innovation Management, vol. 1, ed. H. Bidgoli, 581–600. New York: Wiley.

Mathews, S. 2009b. Valuing risky projects with real options. Research-Technology Management 52(5): 32–41.

Mathews, S. H., Datar, V. T., and Johnson, B. 2007. A practical method for valuing real options. Journal of Applied Corporate Finance 19(2): 95–104.

O’Connor, G. C., and Ayers, A. D. 2005. Building a radical innovation competency. Research-Technology Management 48(1): 23–27.

Paulson, A. S., O’Connor, G. C., and Robeson, D. 2007. Evaluating radical innovation portfolios. Research-Technology Management 50(5): 17–29.

Robeson, D., and O’Connor, G. C. 2007. The governance of innovation centers in large established companies. Journal of Engineering and Technology Management 24(1–2): 121–147.

Rosenø, A. 2008. Developing radical innovation capabilities in established firms. A paper presented at Front End of Innovation Europe, Vienna, January 31.

Say, T. E., Fusfeld, A. R., and Parish, T. D. 2003. Is your firm's tech portfolio aligned with its business strategy? Research-Technology Management 46(1): 32–38.

Scriven, E. 2001. Determining a project's probability of success. Research-Technology Management 44(3): 51–57.

Terwiesch, C., and Ulrich, K. 2008. Managing the opportunity portfolio. Research-Technology Management 51(5): 27–38.