Matching Systems Engineering Process Improvement Frameworks/Solutions with Customer Needs
Definition: Frameworks that enable systems engineering process improvement provide a basic conceptual structure to solve or address complex issues by designing, establishing, refining, and forcing adherence to a consistent design approach .
Keywords: business performance model, improving efficiency, organizational maturity, process-driven management, process improvement, quality management, systems engineering best practice
MITRE SE Roles & Expectations: MITRE systems engineers (SEs) are expected to collaborate with government and contractor organizations to select and tailor systems engineering process improvement models—e.g., Software Process Improvement and Capability Determination (SPICE), Software Engineering Institute (SEI) Ideal, Capability Maturity Model Integrated (CMMI), and Lean Six Sigma, etc. These are used to modify, integrate, test, baseline, deploy, and maintain systems engineering processes for the government acquisition and/or contractor organizations . SEs should be aware of the spectrum of choices for continuous process improvement (CPI) efforts, be able to form recommendations about them, and assist in implementing a selected approach within their work environment.
Each process improvement framework brings its own set of standards and strengths to satisfy customer needs. Some, such as CMMI, Control Objectives for Information and related Technology (COBIT), and Information Technology Infrastructure Library (ITIL) come from a set of best practices. Others, such as Lean Six Sigma, consist of strategies or tools to identify weaknesses and potential solutions. Because systems engineering process improvement frameworks overlap, more than one framework may match the customer needs for a CPI effort. These frameworks are available for any effort; there are no exclusivity rights. For example, the ITIL characterizes itself as "good practice" in IT service management. The SEI has a CMMI for services .
There are standard sets of processes that provide templates and examples of key processes such as IEEE's ISO/IEC 15288 . The Six Sigma, Lean Manufacturing, Lean Six Sigma family of frameworks each contain tools to assess and improve processes, and are currently in use in government organizations.
CMMI has gained a great deal of popularity over the past few years. The Government Accountability Office (GAO) has been basing its oversight reviews on this framework and the results are flowing back to the departments and agencies with references to CMMI best practices. As a result, some in government are taking the view that an organization aligned with CMMI best practices and certified for its software development processes' level of maturity at 2 or 3 will receive greater approval from the GAO and other oversight groups. This has promoted CMMI's influence.
Lean Six Sigma is growing in popularity. It represents the joining of Six Sigma and Lean Manufacturing. Six Sigma was heavily touted a few years ago in government and industry and is still used in some sectors because of the methodology's success in eliminating defects. However, the downside was that it took too long and was too expensive. Lean Six Sigma, as the name implies, is faster to complete and requires fewer resources. The combination of Six Sigma and Lean tools and techniques is more effective and efficient and contains a richer solution set.
Selecting a framework may be based on factors that do not relate to the problem being addressed. Popularity of the framework can come into play. The background and experience of the individual leading the CPI effort can influence the approach. The customer may have some predetermined ideas as well.
Matching a process improvement framework/solution to the customer needs involves two issues: working on the systems engineering/technology problem, and developing and executing a strategy to orchestrate any associated organizational change. Any solution will require some members of the organization to perform their duties differently. Continuous process improvement often has a long-term, on-going impact, as the processes are refined. Individuals in an organization executing CPI need to be comfortable with the change and embrace it.
Some of the frameworks are concerned with "what" should be done and others focus on "how" it should be done. Frameworks such as CMMI are in the "what" category. For example, CMMI indicates the need for a requirements development process area. It involves establishing customer and product requirements and analysis and validation. However, it does not prescribe elements such as what approach to use, the process model, or what rules to follow. Frameworks such as Six Sigma are in the "how" category. Six Sigma suggests appropriate tools to arrive at the right solution. Examples for requirements development include house of quality, voice of the customer, and affinity diagrams tools.
There is a high percentage of failure or slow progress in CPI efforts. In a 2007 quarterly report, SEI reported that it takes on average 20 months to attain CMMI Level 2 and 19 months to attain CMMI Level 3. The variation of time to attain Level 2 and Level 3 is large.  Many organizations fail on their first attempt and have to restart before they are successful. This is a consistent pattern with CMMI implementation. If there is frequent change in leadership at the top of the organization, the risk for completion is higher because new leadership often brings new direction.
Best Practices and Lessons Learned
- Consider the path of least resistance. Be a pragmatist but do not give up on principles. There are many ways to meet customer needs. If you have built a trusting relationship, you can guide the way customer needs are met through the appropriate approach. When more than one framework will do the job, do not get hung up on which framework is "best" (e.g., ITIL  versus CMMI for Services ). If there is a positive history of ITIL in the organization and it fills the need compared to a CMMI for services solution, evaluate whether the benefits you might gain outweigh the costs and difficulty of making the shift. When you are assessing alternative approaches, when the more difficult path is the only way to accomplish the client goals completely, then advise the client accordingly and include a clear and frank discussion of the difficulties.
- Critically consider the customer's true need. Beware of falling into the trap of investing time defining a complete program level set of policies, charters, data repositories, metrics, processes, procedures, etc., if the customer really only needs processes specific to each project and has no desire to be certified at any level.
- Organizational change is the difficult step. Implementing CPI normally involves a very different way of doing business in a continuously changing environment. CPI may not be universally viewed as an improvement by those affected by the change. Consider bringing in an organizational change expert.
- Combine approaches. There may be strength in a combined or hybrid CPI approach. As an example, the CMMI framework is built on best practices from a variety of government and industry sources that do a very good job of explaining "what" to do, but do not provide guidance on "how" to do it. For example, for a certain level of maturity, CMMI requires a requirements development process. However, it does not define how to do that process. If you are looking for predefined processes, consider ISO 9000.  If you are looking to create your own, consider Lean Six Sigma and tools like voice of the customer, affinity diagrams, or house of quality.
- Gain and use upper management support. Elicit and gain upper management support to settle on the right framework/solution for the organization before attempting to implement it. This is crucial regardless of which framework is selected. Use a top-down strategy to promote the CPI program. A bottom-up approach alone rarely results in a successful outcome. Even if it is successful, the project will usually take much longer.
- Avoid labeling processes as new. Embed the process improvement effort into the normal way the organization conducts business. Avoid calling attention to it by using the framework name. Make the process part of refining the organization's customary system development life cycle. Otherwise, you risk creating the assumption that workloads will be enlarged. Compliance is more likely when those affected understand the change is merely a refinement to the work they are already doing.
There are many reasons why an organization may not gain traction when adopting a CPI program. When an implementation falters, do not automatically assume it is due to the chosen framework or approach. Ask yourself whether there really is a compelling reason for CPI or if there is a lack of engagement or buy-in by the customer. If so, it may be better to defer until another time.
References & Resources
- Statemaster.com Encyclopedia, accessed February 12, 2010.
- The MITRE Institute, "MITRE Systems Engineering (SE) Competency Model, Section 3.8."
- CMMI Product Team, August 2006, CMMI® for Development, Version 1.2, Carnegie Mellon Software Engineering Institute.
- ISO/IEC 15288 IEEE, February 1, 2008, Systems and software engineering—System life cycle processes, 2nd edition.
- Software Engineering Institute (SEI), 2006, SEI Annual Report, Carnegie Mellon.
- Bajada, S., February 2010, ITIL v3 Foundations Certification Training. This book and foundations courses for the Information Technology Instruction Library (ITIL) are available through The MITRE Institute's SkillPort e-Learning site.
- CMMI Product Team, February 2009,CMMI® for Services: Improving processes for better services, Version 1.2, Carnegie Mellon Software Engineering Institute.
- International Standards Organization (ISO), January 2009, ISO 9000.