Follow Us:

Assessing Technical Maturity

Definition: Assessing the maturity of a particular technology involves determining its readiness for operations across a spectrum of environments with a final objective of transitioning it to the user. Application to an acquisition program also includes determining the fitness of a particular technology to meet the customer's requirements and desired outcome for operations.

Keywords: disruptive technology, emerging technology, mature technology, revolutionary technology, sustaining technologies, technological innovation, technology assessment, technology insertion, technology readiness, TRL

MITRE SE Roles & Expectations: Systems engineers (SEs) are expected to anticipate future technology needs and changes based on a broad understanding of the systems context and environment, recommend long-term technology strategies that achieve business/mission objectives, and exploit innovation. As part of acquisition planning, the ability to successfully procure new technology and systems involves assessing current technology to support the program requirements. Understanding how to assess technology readiness, apply technologies to a program, and mature technologies for insertion are an important part of what MITRE systems engineers are expected to provide our customers. And because MITRE serves a role independent from commercial industry, MITRE is often asked to independently assess a particular technology for "readiness."

Whether assessing the usefulness of a particular technology or research program, or assessing the ability to meet a set of new requirements with mature technology, it is best to first understand the typical cycle technology developments follow and the methodologies to consider for selecting the appropriate path for your program.

Best Practices and Lessons Learned

Technology Hype Cycle: One way to look at technology maturity is through a Gartner hype cycle [1]: a graphic representation of the maturity, adoption, and business application of specific technologies. Gartner uses hype cycles to characterize the over-enthusiasm or "hype" and subsequent disappointment that typically follow the introduction of new technologies. A generic example of Gartner hype cycles is shown in Hype Cycles (Figure 1 below).

A hype cycle in Gartner's interpretation has five steps:

• Technology Trigger: The first phase of a hype cycle is the "technology trigger" or breakthrough, product launch, or other event that generates significant press and interest.
• Peak of Inflated Expectations: In the next phase, a frenzy of publicity typically generates over-enthusiasm and unrealistic expectations. There may be some successful applications of a technology, but there are typically more failures.
• Trough of Disillusionment: Technologies enter the "trough of disillusionment" because they fail to meet expectations and quickly become unfashionable. Consequently, the press usually abandons the topic and the technology.
• Slope of Enlightenment: Although the press may have stopped covering the technology, some businesses continue through the "slope of enlightenment" and experiment to understand the benefits and practical application of the technology.
• Plateau of Productivity: Mainstream adoption starts to take off. Criteria for assessing provider viability are more clearly defined. The technology's broad market applicability and relevance are clearly paying off.

Figure 1. Hype Cycles

Although Gartner references the "press" above, technology hype can and does occur throughout different organizations. It can often result in significant program investment funding being applied to technologies that may not be suitable for the intended system or user, but were deemed promising by program stakeholders. The above listed steps are applicable to all MITRE sponsors and customers to which technology programs are marketed. When significant attention is given by program stakeholders to a new research, technology, technology development program or demonstration, the targeted technology should be objectively evaluated and assessed for maturity as soon as possible before committing any significant program investment funding.

Technology Maturity: A generic depiction of technology maturity is shown by the s-curve in Figure 2. In general, technology can be defined as follows: new technology has not reached the first tipping point in the s-curve of technology maturity; improving or emerging technology is within the exponential development stage of the curve after the first tipping point and before the second tipping point; mature technology follows the second tipping point before the curve starts down, and aging technology is on the downward tail.

Figure 2. Technology Maturity

The most universally accepted methodology for assessing the upward slope of this curve is the Technology Readiness Level (TRL) scale [2]. There are actually several versions of the original NASA-developed TRL scale depending on the application (software, manufacturing, etc.), but all rate a technology based on the amount of development completed, prototyping, and testing within a range of environments from lab (or "breadboard") to operationally relevant. It is critical to get a common and detailed understanding of the TRL scale among program stakeholders, particularly concerning terms like "simulated environment," "relevant environment," and "operational mission conditions" which must be interpreted in the context of the system or capability under development. Close communication among the program office, operational users, and the developer on these terms are needed to ensure an accurate assessment. One factor the current TRL scale does not address is how well the developed technology fits into the architecture and system structure of the program absorbing it. This is an integral part of the systems engineering job and critical to the success of the technology transition.

Selecting Technology Alternatives: For assessing which technology to employ to satisfy new requirements, various fitness criteria can be used to select which alternative will best realize customer desired outcomes from the total spectrum of technologies available. Criteria that consider both the technology and the customer's ability to assimilate it are more likely to succeed than those that consider only the technology (as seen above in the use of TRLs). Moore [3] identifies types of customers as: innovators, early adopters, early majority, late majority, and laggards. The curve depicted in Figure 3 is referred to as the technology adoption lifecycle, or Roger’s Bell Curve [4].

Figure 3. Roger's Bell Curve

As the names suggest, each customer type has its own tolerance for change and novelty. Technology assessment considers the customer's tolerance for disruptive change as well as new or old technologies. For example, it would not be appropriate to recommend new technology to "late majority" customers, nor mature technology to "innovators."

Department of Defense acquisition programs are required to assess all threshold capabilities in the Capabilities Description Document for maturity; those deemed to be met with immature technology (a TRL of less than six) will not be considered further as "threshold" and may jeopardize the program milestone decision. Programs structured to inject developing technologies could be more receptive to innovation and less mature technologies, but in this case be sure to carefully evaluate the risks involved (for further reference, see the Risk Management topic in this section).

ABC Alternatives: Another dimension of the selection criteria considers the capabilities of technology providers. Former Director of the Defense Information Systems Agency, Lt Gen Charles Croom, devised a new philosophy for acquisition called ABC [5]. In the "ABC" concept, "A" stands for adopt existing technology, "B" is buy it, and "C" is create it yourself. Adopt may seem an obvious decision if the technology fits the purpose, but both the technology and the provider should be evaluated for reliability and sustainability. With the buy alternative, vendor responsiveness and capability are concerns (for further reference, see the Integrated Logistics Support topic in this section). Create is the choice of last resort, but it may be the best alternative in certain circumstances.

References & Resources

1. Gartner Hype Cycles, viewed March 2, 2010.
2. "Technology Readiness Level," Wikipedia, viewed February 22, 2010.
3. Moore, Geoffrey A., 1998. Crossing the Chasm, Capstone Publishing Limited.
4. Rogers, Everett, Diffusion of Innovation Model, "Technology Adoption Lifecycle," Wikipedia, viewed February 22, 2010.
5. Gallagher, Sean, 20 March 2008, "Croom: Acquisition Done Better, Faster, Cheaper," Federal Computer Week. viewed February 22, 2010.

Additional References & Resources

1. "Technical Readiness," MITRE Project Leadership Handbook, viewed February 22, 2010.