Assessing Technical Maturity
Definition: Assessing the maturity of a particular technology involves determining its readiness for operations across a spectrum of environments with a final objective of transitioning it to the user. Application to an acquisition program also includes determining the fitness of a particular technology to meet the customer's requirements and desired outcome for operations.
Keywords: disruptive technology, emerging technology, mature technology, revolutionary technology, sustaining technologies, technical risk, technological innovation, technology assessment, technology insertion, technology readiness, TRA, TRL
MITRE SE Roles and Expectations: Systems engineers (SEs) are expected to anticipate future technology needs and changes based on a broad understanding of the systems context and environment, recommend long-term technology strategies that achieve business/mission objectives, and exploit innovation. As part of acquisition planning, the ability to successfully procure new technology and systems involves assessing current technology to support the program requirements. Understanding how to assess technology readiness, apply technologies to a program, and mature technologies for insertion is an important part of what MITRE SEs are expected to provide our customers. And because MITRE serves a role independent from commercial industry, MITRE is often asked to independently assess a particular technology for "readiness."
Whether assessing the usefulness of a particular technology or research program, or assessing the ability to meet a set of new requirements with mature technology, it is best to first understand the typical cycle that technology developments follow and the methodologies to consider for selecting the appropriate path for your program.
Best Practices and Lessons Learned
Technology Readiness Assessment (TRA). NASA originally developed Technology Readiness Levels (TRLs) in the 1970s and 1980s . Other governmental agencies followed in the ensuing decades, including the U.S. Department of Defense (DoD)  and Department of Energy (DOE) , as well as European and international agencies. This systematic, metrics-based process assesses the maturity of, and the risk associated with, critical technologies. DoD's TRA guidance includes a skeletal template for TRAs, comprising a program overview, identification of critical technologies, and an assessment of program technology risks and readiness. It also provides TRL definitions, descriptions, and supporting information.
Technology hype cycle. One way to look at technology maturity is through a Gartner hype cycle : a graphic representation of the maturity, adoption, and business application of specific technologies. Gartner uses hype cycles to characterize the over-enthusiasm, or "hype," and subsequent disappointment that typically follow the introduction of new technologies. A generic example of Gartner hype cycles is shown in Figure 1.
A hype cycle in Gartner's interpretation has five steps:
- Technology Trigger: The first phase of a hype cycle is the "technology trigger" or breakthrough, product launch, or other event that generates significant press and interest.
- Peak of Inflated Expectations: In the next phase, a frenzy of publicity typically generates over-enthusiasm and unrealistic expectations. Some technology applications may be successful, but typically more are failures.
- Trough of Disillusionment: Technologies enter the "trough of disillusionment" because they fail to meet expectations and quickly become unfashionable. Consequently the press usually abandons the topic and the technology.
- Slope of Enlightenment: Although the press may have stopped covering the technology, some businesses continue through the "slope of enlightenment" and experiment to understand the benefits and practical application of the technology.
- Plateau of Productivity: Mainstream adoption starts to take off. Criteria for assessing provider viability are more clearly defined. The technology's broad market applicability and relevance are clearly paying off.
Although Gartner references "the press," technology hype can and does occur throughout different organizations. It can often result in significant program investment funding's being applied to technologies that may not be suitable for the intended system or user but were deemed promising by program stakeholders. The previous steps are applicable to all MITRE sponsors to which technology programs are marketed. When program stakeholders give significant attention to new research, technologies, or technology development programs or demonstrations, the targeted technology should be objectively evaluated and assessed for maturity as soon as possible before committing any significant program investment funding.
Technology maturity. A generic depiction of technology maturity is shown by the s-curve in Figure 2. In general, technology can be defined as follows:
- New technology has not reached the first tipping point in the s-curve of technology maturity.
- Improving, or emerging, technology is within the exponential development stage of the curve after the first tipping point and before the second tipping point.
- Mature technology follows the second tipping point before the curve starts down.
- Aging technology is on the downward tail.
The most universally accepted methodology for assessing the upward slope of this curve is the Technology Readiness Level (TRL) scale . There are actually several versions of the original NASA-developed TRL scale, depending on the application (software, manufacturing, etc.), but all rate a technology based on the amount of development completed, prototyping, and testing within a range of environments from lab (or "breadboard") to operationally relevant. It is critical to get a common and detailed understanding of the TRL scale among program stakeholders, particularly concerning terms like "simulated environment," "relevant environment," and "operational mission conditions," which must be interpreted in the context of the system or capability under development. Close communication among the program office, operational users, and the developer on these terms is needed to ensure a shared understanding. One factor the current TRL scale does not address is how well the developed technology fits into the architecture and system structure of the program absorbing it. This is an integral part of the systems engineering job and critical to the success of the technology transition. TRLs should be tracked over time to ensure that a technology is maturing as expected and, if it is not, to determine whether an alternative technology should be pursued.
Selecting technology alternatives. For assessing which technology to employ to satisfy requirements, various fitness criteria can be used to select which alternative will best realize the sponsor's desired outcomes from the total spectrum of technologies available. Criteria that consider both the technology and the sponsor's ability to assimilate it are more likely to succeed than those that consider only the technology (as in the use of TRLs). Moore  identifies types of sponsor as innovators, early adopters, early majority, late majority, and laggards. The curve in Figure 3 is referred to as the technology adoption life cycle, or Roger’s Bell Curve .
As the names suggest, each sponsor type has its own tolerance for change and novelty. Technology assessment considers the sponsor's tolerance for disruptive change as well as for new or old technologies. For example, it would not be appropriate to recommend new technology to "late majority" sponsors nor mature technology to "innovators." DoD acquisition programs are required to assess all threshold capabilities in the Capabilities Description Document for maturity; those deemed to be met with immature technology (a TRL of less than 6) will not be considered further as "threshold" and may jeopardize the program milestone decision. Programs structured to inject developing technologies could be more receptive to innovation and less mature technologies, but in this case be sure to carefully evaluate the risks involved (see the Risk Management topic in this section).
ABC alternatives. Another dimension of the selection criteria considers the capabilities of technology providers. Former Director of the Defense Information Systems Agency, Lt. Gen. Charles Croom, devised a new philosophy for acquisition called ABC . In the ABC concept, "A" stands for adopt existing technology, "B" is buy it, and "C" is create it yourself. Adopt may seem an obvious decision if the technology fits the purpose, but both the technology and the provider should be evaluated for reliability and sustainability. With the buy alternative, vendor responsiveness, capability, and viability are concerns (see the Integrated Logistics Support topic in this section). Create is the choice of last resort, but it may be the best alternative in certain circumstances. When choosing Create, consider the entire systems engineering life cycle, including operations and maintenance, as this will have an impact on life-cycle cost.
References and Resources
- Banke, J., August 20, 2010, Technology Readiness Levels Demystified, NASA.
- Asst. Secretary of Defense for Research and Engineering, April 2011, Technology Readiness Assessment (TRA) Guidance, Department of Defense.
- U.S. Department of Energy, September 15, 2011, Technology Readiness Assessment Guide, DOE G 413.3-4A.
- Gartner, Inc., Gartner Hype Cycles, accessed October 15, 2015.
- Mai, T., July 31, 2015, Technology Readiness Level, NASA.
- Moore, G. A., 1998, Crossing the Chasm, Capstone Publishing Ltd.
- Rogers, E., Diffusion of Innovation Theory, Indiana University, accessed December 8, 2015.
- Gallagher, S., March 20, 2008, Croom: Acquisition Done Better, Faster, Cheaper, Federal Computer Week, accessed December 8, 2015.