Follow Us:

MITRE FFRDC Independent Assessments

Definition: An independent assessment is a tool that can be used at any point in a program lifecycle to provide insight into progress and risks.

Keywords: audit, baseline assessment, independent expert review, independent technical assessment, red team, SCAMPI appraisal, Tiger Team

MITRE SE Roles & Expectations: MITRE systems engineers are expected to be able to lead or participate in independent assessment teams, particularly when program processes are being evaluated or there are concerns about program progress or contractor performance [1]. MITRE systems engineers are expected to apply strong domain and technical expertise and experience and perform with objectivity consistent with the FFRDC role.

Context

An Independent Assessment is a team activity in which the team leader and team members are not members of the organization being assessed. This allows the team to more readily fulfill its charter objectively and without conflicts of interest. Ideally, assessments are proactive and intended to provide an early look into what potential problems may be on the horizon in time to take action and avoid adverse impact to the program. For example, an assessment may be used to take an early look at the challenges associated with technologies and provide feedback to the technology development strategy. In other cases, an assessment might be intended to assess progress with a process improvement framework, such as CMMI [Capability Maturity Model Integration) [2], or may be intended to examine causes of concerns with program performance.

In MITRE's work, independent assessments are known by several names, including:

• Independent Reviews
• Red Teams
• Blue or Green Teams
• Technical Assessments—Tiger Teams
• Appraisals (against a model or standard)
• Independent Expert Reviews (IER)
• Audits and Compliance Assessments
• Accident Investigations

For related information, refer to the article "Planning and Managing Independent Assessments" in this topic, and  "Data-Driven Contractor Evaluations and Milestone Reviews" and "Earned Value Management" in the topic Contractor Evaluation in the "Acquisition Systems Engineering" section.

MITRE systems engineers are frequently asked to lead and participate in independent assessments because the characteristics of an FFRDC, as chartered under the Federal Acquisition Regulation (FAR) 35.017 [3], promote independence, objectivity, freedom from conflicts of interest, and technical expertise. These characteristics support the management goals of the assessment team.

For example, the FAR describes the special relationship between a sponsor and its FFRDC [3]. The FFRDC:

"... meets some special long-term research or development need which cannot be met as effectively by in-house or contractor resources...to accomplish tasks that are integral to the mission and operation of the sponsoring agency."

"...has access, beyond that which is common to the normal contractual relationship, to government and supplier data, including sensitive and proprietary data, and to employees and facilities."

"...conduct[s] its business...to operate in the public interest with objectivity and independence, to be free from organizational conflicts of interest, and to have full disclosure of its affairs to the sponsoring agency."

Best Practices and Lessons Learned

MITRE's assessment teams have experienced some challenges, including: [4]

• Sponsor oversight has been delegated leading to a lack of clarity about what needs to be accomplished.
• The review is additional work and seen as a lower priority.
• Members of the organization being reviewed are not able to participate as planned.
• Subjects of the review are not prepared for the review.
• Objective evidence is difficult to locate.
• Appraisal team working space is rarely available.

MITRE practitioners have found that these and other challenges can be mitigated through communication and following the process described in the Planning and Managing Independent Assessments article in this Guide.

References & Resources

1. Independent Technical Assessments.
2. The MITRE Institute, September 1, 2007, MITRE Systems Engineering (SE) Competency Model, Version 1, p. 39.
3. Standard CMMI® Appraisal Method for Process Improvement (SCAMPI(SM)) Methodology.
4. Federal Acquisition Regulation (FAR).

Additional References & Resources

Clapp, J.A. and P.G. Funch, March 5, 2003, A Guide to Conducting Independent Technical Assessments, MITRE Center for Air Force C2 Systems.

Kennedy, J. and L. Rosa, Leading an Independent Review Team, The MITRE Institute, Course TSE-501, The MITRE Corporation.

The MITRE Corporation, August 21, 2009, "Assessment & Adaptation," MITRE Project Leadership Handbook.

The MITRE Corporation, August 21, 2009, "Customer and Contractor Interaction," MITRE Project Leadership Handbook.

The MITRE Corporation, August 21, 2009, "FFRDC Role & Public Interest," MITRE Project Leadership Handbook.

The Project Management Institute, 2008, A Guide to the Project Management Body of Knowledge, (PMBOK Guide), 4th edition.