Definition: A continuous and persistent process that enables decision makers to strategically and operationally manage resources to maximize accomplishment of desired outcomes (e.g., mission results, organizational improvements, enhancement of operational capabilities) within given constraints and constructs such as regulations, interdependent architectures, budgets, concept of operations, technology, and mission threads.
Keywords: capability, capabilities, optimize, outcomes, portfolio analysis, portfolio management, portfolios
MITRE SE Roles & Expectations: MITRE systems engineers are expected to understand and keep abreast of sponsor and customer portfolio management (PfM) challenges, themes, and strategies. They are expected to recommend and apply systems engineering approaches to address PfM opportunities and issues, including data-driven analysis, incremental baseline innovation, time-certain and price-driven agile acquisition, and the exploitation of commercial development methods and products.
Overview of Portfolio Management
Modern portfolio theory provides foundational concepts that are useful in multiple portfolio management environments. Portfolio management is about aggregating sets of user needs into a portfolio and weighing numerous elements to determine the mix of resource investments expected to result in improved end user capabilities. The key elements that portfolio management must assess are overall goals, timing, tolerance for risk, cost/price, interdependencies, budget, and change in the enterprise environment over time.
Accountability for and transparency of government expenditures has been a significant focus during the last two decades. More recently it has become important that these expenditures address key enterprise (agency, mission) outcomes efficiently, effectively, and collectively rather than as independent and unrelated initiatives. Portfolio management is a key tool for supporting this form of fiscal accountability. A simplified overview of portfolio management activities is provided in Figure 1 below. Various laws, directives, and guides relate to portfolio management.
At present there are two major, definitive types of portfolio management: (1) information technology (IT) portfolio management and (2) capability portfolio management (CPM). IT portfolio management deals with investment analysis from a hardware and software perspective for an enterprise: dealing with the configurations and evolution for IT assets, re-capitalization, savings through concepts like regionalization, virtualization, shared assets, cloud capabilities, etc. CPM deals with managing the end user capabilities (applications, data, services) as investment options and selecting the best set of functional capabilities to invest resources in and to evolve over time. Government organizations are currently in various stages of implementation with multiple approaches, and they have met with different levels of success. MITRE systems engineers can use our knowledge to help analyze the best way forward for successful customer IT architectures and implementations, as well as use our knowledge of the operational needs and associated capabilities within the enterprises to help customers performing CPM. There is a place for both types of portfolio assessments. As an enterprise conducts CPM, they will undoubtedly need to construct the best IT environment to support the capabilities.
Stakeholder Engagement, Roles, and Responsibilities
Portfolio management requires leadership commitment. Leadership must endorse portfolio management goals, a rigorous and analytical process, and the willingness to make difficult recommendations and decisions such as investment termination. MITRE can help with leadership commitment by analyzing the options and formulating courses of action that define the best investment and use of resources and by highlighting the cost-benefit and return on investment from the recommended application of resources.
Engage all stakeholders early and often. Due to the significant number of portfolio capability providers, as well as the organizational constructs/governance structures that may divide the decision maker and the portfolio managers, it is important to identify all stakeholders and to understand the magnitude of their stake in the portfolio and how specific stakeholder groups might drive portfolio components and the portfolio. Understanding the different roles, responsibilities, and perspectives of the stakeholders (including those of your particular customer) helps in devising strategies to ensure objective assessment of potential investments, stakeholder buy-in, viable and affordable recommendations, and minimization of "back-door" efforts. Knowing how each stakeholder group drives the portfolio can suggest the needed level of attention that must be paid to each. A minority stakeholder may drive a single requirement that drives solutions to be significantly more complex and costly than would a majority stakeholder holding 90 percent of the requirementsa situation that the government must avoid.
Recommended Tools and Techniques to Use in Portfolio Management
Using the process of Figure 1, the following sections describe the tools and techniques along with the actions of systems engineers to help accomplish portfolio management.
Analysis Tools and Techniques
- Establish the Common Set of Operational Needs Over Time. Understand what requirements, capabilities, goals, or outcomes need to be achieved, when they must be delivered, how they are measured, and how they are prioritized. This information provides specific and common targets for each element of the portfolio. However, the collection of this data is not normally standardized and sustained in a meaningful way in all organizations. The development and maintenance of this information should be a goal of the collective stakeholders of the portfolio. The SEG topics on Concept Development and Requirements Engineering provide articles on how to help manage the concepts, needs, and requirements of users and can understand the priorities of these as input to portfolio management.
- Establish an Analytic Process. The government needs to move away from using forcefully conveyed arguments that appeal to group emotions to using an analytical foundation for portfolio decisions. In order to build a compelling case for portfolio selection (and continuation), help establish a well-understood and transparent analytic process with a framework that addresses specified portfolio goals/outcomes, incorporates key contextual elements (e.g., scenarios) and criteria/metrics important to the decision, utilizes appropriate data sets, and allows consistent comparison across the portfolio options. Execution of this process should provide insight into the tradeoffs and define how the portfolio will be chosen or modified. Figure 2 provides an example of what is called an "efficient frontier" created using MITRE's Portfolio Analysis Machine (PALMA) optimization tool. The efficient frontier showcases the portfolio options (black and red points) that provide the most "bang for the buck" (defined here by an overall benefit measure for a given budget). It also shows gray points that are a subset of the less efficient portfolios. MITRE systems engineers should understand the mechanics and value of using tools such as PALMA to better understand trade-offs and should call upon MITRE experts such as those in MITRE's Center for Acquisition and Systems Analysis and other analysis organizations like the National Security Analysis Group to help with the analyses.
- Be Data Driven. Organizations generally do not have a repository that provides a single, consistent, and robust database to support organizational portfolio management. Based on time and availability, the team (including MITRE systems engineers) should develop the best data set to assess criteria and create consistent comparisons across the investments considered. While the most objective data is sought out, the best data available may actually come from subject matter experts (SMEs). MITRE systems engineers can help facilitate cost-benefit assessments with groups of SMEs by providing questionnaires and facilitated discussions of proposed capability benefits, prioritization, process improvements, resource savings, etc.
- Understand Contents of a Portfolio. A full understanding of the investments in a portfolio, as well as of those that may be related to or impacted by your portfolio, is required in order to define the correct tradeoffs for decision making. A common understanding of the portfolio content by the decision makers, reviewing bodies, stakeholders, and systems engineers is critical. MITRE systems engineers can help lay out the contents of the portfolio, their interconnections, feasibility, risks, etc. Modeling and simulation and in some cases actual prototyping and experimentation can be used by MITRE systems engineers to highlight the features of the critical drivers in the portfolio.
- Determine Dependencies. Current operations require multiple systems, system components, etc., to work together to create appropriate capabilities, which in turn creates dependencies within portfolios that must be identified and understood. Refer to the SEG Risk Management topic for some guidelines on dependency risks. There are multiple data sources, artifacts, policies, etc. that must be considered in total, as well as an understanding of the links across these elements, to gain full comprehension of the complexity of the portfolio. For example, we need to ensure that we understand the connection between elements such as requirements, capabilities, mission threads, architectures, and systems.
Selection Tools and Techniques
- Know the Baseline. Based on an understanding of user needs, the team must understand how current needs are being met before recommending changes to the portfolio. In some cases this is called development of the "baseline." MITRE systems engineers help establish the baseline using techniques like federated architectures (see the SEG article Architectures Federation) where the baseline and subsequent evolution of the proposed portfolio can be captured.
- Adequacy of the Options. A robust set of options allows key insights into the tradeoffs and their drivers to address portfolio off-set drills and changes in funds. Various levels of the options may be addressed including alternate acquisition strategies, different risk profiles, and different cost-effectiveness profiles. MITRE SEs can help the customers understand the pros and cons of each option including feasibility, risk, performance, cost, and schedule considerations.
Control Tools and Techniques
- It's More Than Technology. The business and programmatic aspects (including cost, acquisition strategies, business models, and risk) of the entire portfolio and its components are as important as the technical aspects.
- How to Buy Is as Important as What to Buy. Defining options within the portfolio should include consideration of how the option should be acquired with consideration of timing and cost. MITRE systems engineers typically help acquisition organizations with strategies and methods and help them extend and apply this knowledge at the enterprise and its portfolio level.
- Establish an Integrated Master Schedule (IMS). An IMS with a life-cycle view is essential to reflect key technical and programmatic interdependencies of portfolio components and allows for focus on synchronization and integration points. System integration analyses can be performed by MITRE systems engineers to determine the schedule dependencies, impacts of slippages, ability to synchronize with other efforts, and assessment of when the portfolio needs to be re-assessed based upon schedule challenges. See the articles Identify and Assess Integration and Interoperability (I&I) Challenges and Develop and Evaluate Integration and Interoperability (I&I) Solution Strategies for insights on systems integration.
Evaluation Tools and Techniques
- Establish Outcomes for the Portfolio and Appropriate Metrics to Monitor Progress. Metrics are difficult to establish, in part because they must reflect common recognition of outcomes across the portfolio. But they are critical to measuring and tracking efficiency and effectiveness. MITRE systems engineers can help formulate metrics through knowledge of the enterprise's operational concepts, needs, requirements, mission accomplishment and assurance, and the operational and technical tradeoffs for these needs.
- What's the Value Proposition? Each investment, program, or resource used must determine its mission as well as how it supports the outcomes/products of the portfolio in which it resides. The cost and funding profile, effectiveness, timeliness of delivery, and risks of each component in relation to the portfolio must be understood. MITRE systems engineers usually focus on the results of getting capabilities to the end users to conduct this mission. Having this knowledge and emphasis and encouraging this perspective across the portfolio stakeholders will help keep the emphasis on the users' value.
Issues and Challenges Impacting Successful Portfolio Management
MITRE system engineers should understand the big picture when it comes to portfolio management and ensure that appropriate perspectives, information, analysis, and tools are brought to bear on the issues.
MITRE systems engineers should understand where the system, system of systems, enterprise, and organization they support fits in the relevant portfolio(s); how it impacts or is impacted by the portfolio investment decisions and its elements; what overarching outcomes need to be supported/achieved and why; and the statutory, regulatory, and policy environment affecting the portfolio decisions for the enterprise. MITRE systems engineers should ensure that appropriate analytics, tools, data sets, and strategies are brought to bear and that appropriate consideration is paid to stakeholders.
The issues related to portfolio management follow:
- Dueling Directives. There may be multiple directives within portfolio management that must be understood in the context of your program or portfolio. These directives may have come from various governing organizations and have conflicting and inconsistent guidance that is difficult to apply to the portfolio assessment. Knowing how your program, portfolio, and/or organization fits into one or more of these structures should help identify these disconnects and support your work to ensure progress and appropriate accountability. MITRE systems engineers can help highlight the inconsistencies and work with the responsible organizations to provide clear and consistent guidance and governance to the whole enterprise.
- Multiple Taxonomies. For many government organization/agencies, there may be multiple taxonomies that define the portfolio structure. Typically, a single taxonomy has not been adopted nor has an approach been developed to allow the taxonomies to be used together effectively to support the goals of portfolio management. Given this, the MITRE systems engineer may need to map across multiple taxonomies to correlate equivalent or similar perspectives/areas of interest.
- Budget Authority. Budget authority may not rest with the portfolio manager, making the portfolio manager more of an advisory resource than a decision maker. The Clinger-Cohen Act suggests IT budget authority rests with the Secretary, the CFO, and the CIO of the particular federal department. In the Department of Defense, for example, budget authority generally resides with the Military Services (Title X) and not the capability portfolio managers. In cases where the portfolio manager also has budget authority, many times the execution of the investment plan can be streamlined. In cases where the responsibilities are in separate organizations, MITRE staff can help the portfolio managers create a persuasive case of the preferred portfolio and highlight the value/cost-benefits of applying the portfolio resources needed.
- Budget Process. In many government agencies, budgets are planned for and executed at a lower level than a portfolio (e.g., program, program element, budget line, appropriation). This adds complexity, making portfolio management execution more difficult because the change recommendations may be at a more granular level than is reflected in budgetary accounting. MITRE systems engineers can maintain the investment profile using the detailed, data-driven analyses previously described and performing sensitivity analyses of changes to the individual components that comprise the portfolio.
- Culture. Portfolio management is a "greater good," or enterprise process, and is not supported within a program acquisition culture rewarded for individual program success rather than enterprise success. This is partly because it takes up-front investment to achieve a longer-term 'greater good' outcome. In addition, the mission success or portfolio savings benefits from portfolio changes are not adequately accounted for or attributed to the portfolio changes, making change a difficult proposition. MITRE systems engineers can demonstrate the greater good by presenting the value that the portfolio choice provides to the enterprise's concepts/needs, federated architecture, business planning, collective integration, design and development activities, and life-cycle cost.
- Political Factors. Politics has consistently been an element of investment decision making, as has operational, economic, technical, and other factors. There may be cases where certain solutions are technically elegant and affordable but not politically feasible. It is important for MITRE to ensure consideration of all factors within the decision-space and to understand the implications of each. Substantive and validated analysis can illuminate whether an investment supports desired portfolio outcomes.
- "Pet Rocks." In some cases, particular solutions may be favored by leadership. This may result in a less valuable investment being selected, which can undermine the ability to secure the most cost-effective portfolio. Analysis can and should inform an understanding of the "value proposition" of investments; however, it may be trumped by leadership preferences. MITRE systems engineers may recommend a solution that is not acted on for reasons outside of their control. When these situations arise, MITRE systems engineers should continue to highlight the risk and to provide an independent view of the situation while helping the government plan and execute their selected alternative.
- Poor Life-Cycle Cost Analysis. Cost overruns are rampant in the government. This is partially due to the low levels of confidence inherent in the original cost estimates of the individual programs. Portfolio management further complicates these already "narrow" cost analyses by altering the foundational assumptions in the cost estimates. For example, a new innovation or a new approach to capability delivery may affect the development or sustainment costs of entire suites of investments. Integrating the effects of portfolio changes on the initial projected life-cycle cost estimate is confounded by flaws in the original cost estimates and by their inability to accommodate the PfM innovation. Refer to the SEG article Life-Cycle Cost Estimation for practices on cost estimating that could be evaluated for use across portfolios.
References & Resources
Government Guidance, Policies, Regulations
- 40 U.S.C. 1401 et seq, Clinger-Cohen Act of 1996, Division E, National Defense Authorization Act for FY1996.
- Air Force Instruction 33-141, Air Force Information Technology Portfolio Management and IT Investment Review, 23 December 2008.
- CJCSI 8410.01, 22 Jun 07, Warfighting Mission Area Information Technology Portfolio Management & Net-Centric Data Sharing.
- Department of Defense Chief Information Officer Desk Reference, August 2006, Foundation Documents, Volume 1.
- Department of Defense Directive 7045.20 USD(P), 25 September 2008, Capability Portfolio Management.
- Department of Defense Directive 8115.01, 10 October 2005, Information Technology Portfolio Management, ASD(NII)/DoD CIO.
- Department of Defense Directive 8115.02, 10 October 2005, Information Technology Portfolio Management Implementation, ASD(NII)/DoD CIO.
- Federal CIO, Capital Planning, and IT Management Committee, Smart Practices Subcommittee, Performance Management Area, "A Summary of First Practices and Lessons Learned in IT Portfolio Management," 2002.
- GAO-04-394G, Information Technology Investment Management, A Framework for Assessing and Improving Process Maturity, March 2004, U.S. Government Accounting Office, Executive Guide, Version 1.1.
- GAO-07-388, An Integrated Portfolio Management Approach to Weapon Systems Investment Could Improve DoD's Acquisition Outcomes, March 2007, U.S. Government Accounting Office, Report to the Committee on Armed Services, U.S. Senate.
- Health and Human Services Policy for IT CPIC.
- Net-Centric Functional Capabilities Board Endorsement of Capability Delivery Increments (CDI) Document, 22 June 2009, The Joint Staff, Memo.
- OMB Circular A-11, August 2009, Preparation, Submission, and Execution of the Budget; Exhibit 53, Agency IT Investment Portfolio.
- OSD Memorandum, 12 January 09 (predecessor: 14 February 2008), Joint Capability Areas.
Journal articles, technical notes, white papers
- Enterprise Scale Portfolio Analysis at the National Oceanic and Atmospheric Administration (NOAA).
- Markowitz, H., March 1952, "Portfolio Selection," The Journal of Finance, Vol. VII, No.1.
- Bhadra, D. and F. Morser, 26-28 September 2005, "Analysis of System-wide Investment in the National Airspace: A Portfolio Analytical Framework and an Example," American Institute of Aeronautics and Astronautics, 5th ATIO/AIAA Conference.
- Integrated Architecture-Based Portfolio Investment Strategies.
- Maizlish, B., Handler, R., 2005, IT Portfolio Management Step-By-Step, John Wiley & Sons, Inc., Hoboken, NJ.
- Moynihan R., Investment Analysis using the Portfolio Analysis Machine (PALMA) Tool.
- Sanwal, A., April 2007, Optimizing Corporate Portfolio Management: Aligning Investment Proposals with Organizational Strategy, John Wiley & Sons, Inc., Hoboken, NJ.
Additional References & Resources
"Program Formulation & Project Planning," MITRE Project Leadership Handbook, viewed 22 February 2010.