Problem
The safety, quality, and cost of healthcare could be improved if those who lack medical expertise had more accurate and timely ways to assess the clinical judgment and billing practices of providers. To fight “fraud, waste and abuse,” payers have traditionally combined statistical analysis with manual clinical reviews of high-cost patients to judge whether treatments were evidence-based and billing practices were appropriate; however, these quality-assurance efforts occur after treatment has been initiated. Currently, patients and payers, including the Centers for Medicaid & Medicare Services (CMS), are relying on retrospectively calculated, population-based clinical quality measures to define medical necessity and assure quality. Like the “pay-and-chase” approach to fraud, retrospectively counting clinical processes after the treatment has been initiated does not prevent suboptimal care. Using population-based measures to compare and differentially pay providers is potentially harmful if the measures do not reflect differences in each provider’s unique patient population and the latest scientific evidence. 

Objectives
The use of  clinical decision support (CDS) services  by clinicians and payers before treatments and payments are executed has been shown to drive patient and provider choices toward more evidence-based and cost-effective use of diagnostics and therapies. The goal of this research team is to demonstrate to CMS the value and feasibility of using CDS-derived metrics of provider performance to advance its pay-for-performance policies and fraud prevention programs. CMS could leverage data, clinical guidelines, rules logic and processes related to point-of-care CDS services to influence practice patterns and improve the clinical relevance of retrospective performance measures. By sharing CDS service costs with providers, private payers, and even patients, CMS could also lower its costs for assessing medical evidence, defining the standard of care, and collecting and storing data for quality assurance and payment purposes.

Activities
We are collaborating with CDS providers, payers, and standard-setting bodies on a number of activities, including a demonstration to CMS of a framework for how to apply CDS and strategies for integrating CDS and CQM policies, standards, and information technology.

Impact
Incorporating CDS into existing quality assurance and billing systems has the potential to prevent suboptimal care by informing clinicians and patients about individualized and cost-effective treatments, which will better align pay-for-performance policies with medical evidence and patient outcomes and minimize “waste and abuse” in billing practices.

Public Release No: 13-0869

Problem
Many federal agencies adjudicate cases, including claims for benefits and services as well as payments to and from the government, through a labor intensive process involving detailed evaluation of each application.  This process unnecessarily varies across agencies, is often inefficient and time consuming, and adversely affects timely response to citizens and other stakeholders.

Objectives
Using the Department of Veterans Affairs as a case study, this team will analyze historical claims data to identify attributes and develop models to predict the probability of a successful or legitimate claim (or conversely, assess the risk of an unacceptable claim).  The model can then be used to expedite payments in the future with additional document collection later in the process or on a random audit basis.

Activities
Step 1 – Explore Business Problem Step 2 – Build Modeling Database Step 3 – Explore Data Step 4 – Prepare Modeling Data Step 5 – Develop Model Step 6 – Model Evaluation Step 7 – Present Model

Impact
The expected outcomes of this research are to significantly reduce the number of days a citizen has to wait for adjudication, improve citizen satisfaction, and increase efficiency and reduce the cost of case processing. We will quantify the benefits of taking a risk-based approach to case adjudication for veteran’s benefits claims to illustrate the impact this approach can have; for example, if 50 percent of benefit claims can be processed 50 percent faster using the actuarial model, the average days to process aclaim will go down by 45 days.

Public Release No: 13-0697

Problem
DHS has sponsored the creation and maintenance of large-scale, high-fidelity models of critical infrastructure systems (e.g., the National Infrastructure Simulation and Analysis Center) to meet the need to determine protection, mitigation and response measures. These tools typically have been extremely expensive and often disappointing in their ability to produce useful results quickly enough to meet program planning or incident response requirements.

Objectives
We will develop and test accelerated systems engineering (SE) techniques specifically designed to more expeditiously employ modeling and simulation (M&S) tools to evaluate alternative CIP protection, mitigation and response measures. Examples of accelerated systems engineering techniques are “agile” software development processes and MITRE’s Accelerated Process Development (APD) tool. These and other accelerated development approaches will be considered.

Activities
We intend to first establish the project foundation (governance, processes, information, and tools) and then develop CIP application scenario(s) and an experimental framework. We will also perform and document experiments to test accelerated processes against the standard process, for the selected CIP application scenario(s) and response models.

Impact
The result of this project will provide DHS with an enhanced capability to evaluate alternative CIP measures, both for program planning and incident response.

Public Release No: 13-0676

Problem
The Health Insurance Portability and Accountability Act of 1996 (HIPAA) defines a safe harbor method for creating releasable de-identified unstructured (free-text) records by first removing 17 identifier types. Current state-of-the-art automated de-identification systems, including our open-source MITRE Identification Scrubber Toolkit (MIST), can de-identify unstructured data at high accuracy; however, de-identified medical records are rarely shared in practice because of privacy concerns, and then only via limited-use two-way agreements, slowing the progress of secondary use research.

Objectives
Our goal is to enable data stewards to share unstructured data for secondary use while maintaining patient privacy. Replacing Protected Health Identifiers (PHI) with synthetic but realistic surrogates (resynthesis) enhances the protective effect of de-identification by causing missed PHI to blend in with the surrogates (Hiding in Plain Sight).

Activities
Together with our collaborators at Group Health Cooperative and Vanderbilt University, who are funded by an NIH grant, we are exploring HIPS. Thus far we have conducted a preliminary experiment to assess the extent to which HIPS prevents human readers from detecting de-identification errors. We will also explore the extent to which HIPS prevents data mining hackers from launching re-identification attacks based on de-identification errors, and will use the results of these investigations to refine the resynthesis component in MIST to enhance HIPS protection. Finally, we have identified partners to take over MIST stewardship and make it available for enterprise use.

Impact
We have already transitioned MIST to numerous external partners who are using it to enable data sharing for clinical studies in several domains. Greater data sharing via de-identification will lead to accelerated advances in health and biomedical research. Transitioning stewardship of MIST will ensure this technology remains available.

Public Release No: 13-0432

Problem
In the past few years, the number of policy documents, initiatives, and summits related to advanced manufacturing is striking and reflects an increasing recognition of the fundamental linkages between advanced manufacturing and the economic and national security interests of the United States.  As a systems engineering FFRDC, it is critical for MITRE to maintain an awareness of the technologies and processes associated with advanced manufacturing and understand their impacts on the systems engineering needs of our sponsors.

Objectives
In this research we will develop a strategy for MITRE engagement in advanced manufacturing that includes recommendations for MITRE’s IR&D portfolio, investments in advanced manufacturing infrastructure,  participation in advanced manufacturing private-public partnerships and consortia, and opportunities for future sponsor engagement. Our  goal is to demonstrate a set of formal relationships among system verifiability, manufacturability, and multiphysics modeling and simulation frameworks within a general manufacturing engineering domain.

Activities
In FY12, the team worked to build a MITRE community of interest in advanced manufacturing and to conduct broad surveys of the ongoing initiatives in advanced manufacturing.  We will continue these activities in FY13, while we also identify the gaps in extant modeling and simulation support to multidisciplinary, multiphysics systems engineering in the advanced manufacturing context.  As a complementary test case, we are examining the design tools and methods under development for DARPA’s Adaptive Vehicle Make program.

Impact
We have completed a first-order assessment of the research landscape in advanced manufacturing and have drafted roadmaps for MITRE engagement in the areas of: synthetic biology and biomanufacturing; nanotechnology; photonics; robotics; modeling, analysis and optimization; systems engineering; and supply chain risk management.  We anticipate that manufacturability and system verifiability will become important to our sponsors.

Public Release No: 13-1148

Problem
A paradigm shift in systems analysis via modest computational resources formulates the need for transformational systems engineering.  On the other hand, one of the open problems in the context of financial stability is identifying anomalies or leading indicators for alerts.  Finding anomalies for alerts has relevance to many cross-cutting problems in fraud, compliance, and financial system integrity as well as in healthcare.

Objectives
A new systems engineering framework, based on metamodeling, aims to rapidly derive an approximated map between the input and output data obtained from large-scale simulations of complex systems. Our financial stability study employs aggregation and visualization of large, disparate time series data pertaining to diverse sectors of the financial system.  The purely data-driven metamodeling technique, being comprised of complementary methods, enables examining “big data” for identification of leading indicators for critical events.

Activities
The key systems engineering activities pertain to developing metamodels for distinct scenarios and building a generalized procedure for developing metamodels for real-time decision analysis.  Other activities pertain to examining financial data from heterogeneous sectors using integrated analytic methods to identify characteristic signatures of financial instability. 

Impact
The merit of metamodeling has been examined in the context of three distinct scenarios in which a structured approach for metamodeling has been developed. The aggregation of financial time series data with widely different time resolution are used to help decision makers understand signatures for catastrophic events.  The outcomes of both systems engineering and financial stability tasks are being discussed with potential customers and end-users for transition opportunities as well as presented at major conferences.

Public Release No: 13-0622

Problem
The Veterans Health Administration's clinical and administrative operations are hampered by a lack of empirical data to guide process and systems redesign decisions.  Clinics initiating projects to improve patient care have no clear picture of their clinic workflow and resource usage.  The VA needs a way to establish an objective baseline against which to plan system changes and measure gains from operational improvements, as well as objectively collect information to guide realignment of work flows and resources for improved care delivery and reduced costs.  These "realities" were highlighted in a February 2012 U.S. Government Accountability Office report.

Objectives
In partnership with Veterans Integrated Service Network (VISN)-1 and the New England Veterans Engineering Resource Center (NE VERC), MITRE will identify and potentially pilot a method or process to understand how clinical providers allocate time between direct patient care and administrative tasks. 

Activities
We will first iteratively develop detailed concepts, analyze requirements and constraints, and refine our technical approach. We will then identify a pilot clinical area of interest to VISN-1 to explore this idea and determine the current state (“As-Is”) of activities by analyzing clinical staff tasks/activities. Next we will identify opportunities to leverage technology and process improvements and develop recommended approaches (“To-Be”) for VISN-1 to use as an objective basis for improving its resource allocation planning.

Impact
We will assist the VA New England Healthcare System to improve its resource allocation process by developing a methodology and pilot to accurately identify the “current state” of clinical staff daily tasks/activities.  This information will provide VISN-1 leadership with objective, quantitative data and recommended approaches on which to base future resource decisions. 

Public Release No: 13-0117

Problem
The U.S. government collects large amounts of data from financial institutions across the nation. The data is a mix of structured, semi-structured, and unstructured reports containing information about people, financial transactions they’ve made, and links between them. While many people appear in the data only as a single instance, there are some people who appear multiple times. In addition, these people may be found in multiple data sources. The challenge for the analyst is to recognize patterns of activity that are indicative of various types of crime. Typically, they would query the structured data provided in reports, collect a set of names from the text often included as part of the report, query on those names, and repeat the process. Their goal is to find groups of people working together to commit fraud--in spite of the tedious manual process of rummaging through a mixture of data. Thus, analysts need help in recognizing financial crime patterns in data, automating the search for relevant data, and putting together a revealing picture of people and the links among them while they commit fraud.

Objectives
The goal of this project is to show how graph analysis techniques on a nosql database can be used to look for known patterns of fraud. The idea is to merge relevant records about people and transactions coming from multiple data sources, put them into a graph database, and use graph analysis techniques to detect relationships among people. This will speed the work of an analyst considerably in recognizing financial crimes from a collection of structured, semi-structured, and unstructured data.

Activities
For this research, we have acquired financial transaction data from FinCEN, which is held at a secure lab at MITRE. Using this data, we are planning to: --Prepare data for analysis, clean the data, and use MITRE software to extract the names of people, organizations, locations, and transactions from unstructured text. --Collect relevant entities and links. Match mentions of people based on their names, addresses, and selected additional fields. We will save the collection of mentions matched as an entity; then, for each mention, we will collect information about financial links to others. --Build a graph and ingest the entities and links into a nosql database to build a graph for analysis. --Detect patterns, using a generic graph querying language, indicative of a financial crime. --Gather results and filter data by showing rings of people working together based on patterns found. --Enhance visualization through display a graphs showing the results.

Impact
This work will provide an infrastructure and a workflow that allows analysts to do cross-data and cross-data type (narrative and fixed fields) analysis to find people or situations that reveal a financial crime. This will help analysts to more quickly and accurately find the “needle in the haystack” from large amounts of financial data.

Public Release No: 13-0327

Problem
The current state of the practice for evaluating the effectiveness of therapeutic methods on upper limb impairments (particularly in recovery from amputation and stroke) is highly subjective and prone to inaccuracy.  Without an accurate metric of therapeutic performance, recovery takes longer, costs more, and is not as effective on average.

Objectives
The ARMS project team is developing a system for collecting a quantitative metric of arm usage over extended timeframes (days or weeks). The system continuously analyzes data from a sensor mounted on the impaired arm of a patient and streams the results to a remote server for presentation to the caregiver.  It leverages sensing, mobile computing, machine learning, and data visualization fields, and will allow caregivers to immediately see the effects of therapeutic actions. 

Activities
We are completing a data collection exercise and validating the system against it.  This involves instrumenting test subjects while they execute tasks designed to simulate Activities of Daily Living.  The data is then annotated on a frame-by-frame basis with information indicating how the subject was using his or her instrumented arm. After this process is complete, we will validate the classification performance of the system on the recorded data by comparing classifier results to annotations. To view the classifier results, we will have visualization available to the medical practitioners. Paired with this analysis is ongoing development of a wrist-mounted sensor prototype using an Inertial Measurement Unit, which measures both linear acceleration and angular velocity. We will also explore alternate sensor modalities and develop the final sensor prototype, designed for field deployment. 

Impact
We have collected data from 27 human subjects and begun  annotation.  We have also continued to refine the classification model, as well as the software pipeline that will deliver data from the field to the caregivers. Our research will provide caregivers with a valuable tool that should drive down the cost of upper limb rehabilitation.

Public Release No: 13-0425

Problem
Monitoring our critical infrastructure (CI) is quite hard.  Understanding what the signals emitted by our CI mean can be even more difficult.  To further complicate matters, our CI is made up of a number of different networks that are interconnected in both obvious and subtle ways.  Therefore, true understanding of the state of one component of our CI requires understanding the state of many other components.  Due to these interconnections and the highly heterogeneous and dynamic nature of the CI, many traditional statistical methods break down or can obscure the actual state of the CI. 

Objectives
The Analyzing the National Critical Infrastructure as a Complex Adaptive System (CICAS) MIP project is exploring the use of tools from Complexity Science to address the problems outlined above.  There appear to be general features of complex systems (which our CI is) that can be used to understand the state and stability of said systems.  These features can be found in the fractal and multi-fractal spectra of the emitted signals, in the network structure(s), in the distribution of frequency/event-size, and in the so called critical slowing down dynamics of the system as it approaches an abrupt change.  In FY13 CICAS is working to implement a number of these algorithms and test them against CI signals to create an early warning system for use in monitoring our CI.

Activities
In FY13 the team will implement and test against CI signals the critical slowing down metrics, fractal and multi-fractal spectra, and Lacunarity analysis.

Impact
If successful, CICAS will give our sponsors charged with monitoring and protecting the CI a new set of tools specifically designed to create new insights into the dynamics and stability of our CI.

Public Release No: 13-0655

Problem
Measuring the integrity and security posture of a mobile device is challenging. Smart phones and tablet computers lack a number of traditional security features that have emerged for commodity laptops and desktops. In addition to restricted power, mobile devices generally do not have a Trusted Platform Module (TPM) and currently lack ubiquitous virtualization capability. Even as these security mechanisms reemerge for mobile devices, no single one is expected to be a “silver bullet.” A defensive strategy can include gathering evidence of the side effects caused by malicious activity. Measurements can be obtained on-device while the main OS is running, externally via USB interfaces, or by placing the phone into a minimalistic recovery state where a smaller execution context is used to check the device’s integrity. This project aims to use an established root of trust of the phone’s running kernel as a platform to further measure the integrity of various operating system components during runtime.

Objectives
This project is a continuation of a project in FY12 that focused on development of a remote “smart charger” and methods for using the “smart charger” to measure the integrity of the running kernel. The “smart charger” became the platform for implementing, amongst other techniques, timing-based attestation on Android phones. Further research and development of the “smart charger” and development of timing-based attestation on Android phones are being continued as MOCSI laboratory projects. The Android System Integrity Measurement project’s goal is to further identify dynamically loaded operating system components that can be measured based on the assumption that a root of trust has been established.

Activities
We will first prioritize the various operating system components as targets for measurement. Each dynamic component then needs to be investigated for valid methods of comparing those components against a previously recorded measurement value. As an example of this activity, assuming a root of trust has been established by some previous means, we will investigate how to measure the zygote binary in memory while running. The Android zygote process is a virtual machine, and links in all core libraries at start-up. When a new process is started, zygote is forked, forming 2 VMs. This important process has been prioritized for measurement because of the potential impact that any changes in memory can have to subsequent application executions. Further activities include participation in a sponsor pilot using FY12 results to verify the integrity of various phone images (e.g., boot.img, recovery.img). Also, tasks are planned to assess the use of “root” privileges versus the use of “non-root” privileges for system integrity and how measurement frequency affects integrity.

Impact
If successful, we can advance sponsors’ ability to inherently trust the running kernel and various loaded operating system components upon which higher level phone services depend. This will have impact on mobile applications that will be able to use this established trust to further trust the integrity of their mobile client’s measurements.

Public Release No: 12-5139

Problem
In previous research, we developed a capability to compose combinations of exploits that describe money laundering schemes. In 2013, we will expand this capability in several different ways to support a more expansive set of problems. The first is to account for adversarial response. The tool uses game theory to account for an adversary’s circumvention of countermeasures and identifies where to put countermeasures so that the adversary cannot improve their situation. Normally this would involve many iterative optimizations, each taking a long time to solve.  A method was found that collapses the game theory problem into a standard optimization problem, contributing to a large speed up.

Objectives
Money laundering uses states that are all of one type. The tool is being expanded to apply to schemes of multiple types, which is important in representing more complex crimes and attacks such as those involving fraudulent practices or technologies. In connection with this tool,  a way to encode fraudulent schemes was developed, and we generated a database of cases that will give users the ability to generate and evaluate different fraud schemes.

Activities
We are embedding more sophisticated methods into the tool to allow scaling to large problem sets, which will circumvent memory and speed issues.

Impact
Our software interface will allow users to put characteristics of a perpetrator into a system before generating the possible schemes. After the schemes are generated, they can be ranked to determine which are of greatest concern.  

Public Release No: 13-0132

Problem
Using commercial cellular networking and handset technology offers significant cost and performance benefits over current military radios. One such technology, LTE (which stands for Long-Term Evolution) is emerging as the global standard for cellular communications. However, since it is commercial technology, its vulnerability in tactical scenarios needs to be addressed. In particular, its susceptibility to electronic warfare and electronic attack (EW/EA) must be assessed and counter-measures developed where necessary and possible.  Further, because of cost considerations, solutions that do not require handset modification are particularly attractive.

Objectives
There are many areas of LTE that present potential EW/EA targets. We have identified these areas through research and will conduct experimentation with LTE signals in the lab using commercial 4G LTE handsets to reveal the actual susceptibility to specific EW techniques.  As these threats are defined, we will develop techniques to combat them (if possible) using mechanisms in the standard and novel methods specific to DoD use, focusing on eNode-B and network architecture modification rather than handset customization. In conjunction with antijam techniques, we will evaluate candidate ideas that use multiple redundancy versions to enable antijam in talk group environments.

Activities
In FY12, the team developed test software and equipment to work with LTE while simulating various electronic threats in an automated test environment. In FY13, we continue to refine our test setup and techniques, most notably by developing software to enable use of all of the physical layer features of our eNode-B simulator to fully assess both jamming and antijam techniques.  At the same time, we are developing and executing further EW tests and developing EA test scenarios. In addition to improving our physical layer simulation capability, we will also use our SPIRENT channel emulator to add the effects of realistic channel propagation to our scenarios. Using the results of these tests, we will identify vulnerabilities in LTE that need to be addressed and investigate the efficacy of methods of fortifying them. First we are looking at mechanisms already present in the LTE standard and, if necessary, will postulate novel methods that do not rely on handset modification.  Our team will continue to work with the LTE Secure Profile Working Group that is responsible for defining the Department of Defense requirements for LTE use.

Impact
Using the testbed developed in FY12, we have already generated some preliminary EW results. In addition to generating data for designing future tests, this also proved the efficacy of our test setup, giving us a solid foundation for our current work. There is currently a strong push to use commercial cellular in DoD space, and our work will support this approach by raising awareness of and quantifying potential tactical vulnerabilities as well as postulating solutions that could enable wider tactical use in more difficult EW environments.

Public Release No: 13-0259

Problem
Under the Affordable Care Act, the Centers for Medicare & Medicaid (CMS) is required to implement the Hospital Value-Based Purchasing (VBP) program, which is intended to reward hospitals based on the quality of care they provide. While CMS has a system in place to evaluate the hospitals based on quality measures, it does not have the means to evaluate the impact of the Hospital VBP program on the U.S. healthcare system as a whole.

Objectives
We will perform an analysis leveraging the Reference Health Services System (RHSS) framework and the Long-term Inter-Industry Forecasting Tool (LIFT)Healthcare economics model to assess the impact of the Hospital VBP program on the U.S. healthcare system in terms of quality and cost.

Activities
The workflow and resources applied to hospital services will be described using the RHSS framework.  VBP metrics will be applied and anlyzed using the multi-scale hybrid modeling approach that we developed and published in the FY12 program. We have developed descriptors and an influence diagram of the impact of Hospital VBP on CMS hospital inpatient payments (in the context of the the many potential impacts on hospital finances and business plans as a complex systems challenge).  

Impact
The capabilitites developed in ths research will facilitate better coupling of agency plans and programs with the healthcare sector economic outlook--impacting wellness of the populace, jobs, income, and GDP (regional and national). Our research will also support program planning and analysis, policy analysis, and management in CMS and the VA.  

Public Release No: 13-0677

Problem
A big challenge at MITRE continis how we can easily share information with our sponsors, customers, and partners in a trusted way. The research goal of this project is to leverage and extend emerging open standards to support information sharing, co-engineering, activity notifications, networking, and access to services available in a multi-organizational environment, in a trusted way. 

Objectives
We will design the overall open standards multi-organizational sharing architecture and explore components to address the various integration layers, including an aggregated notification service, a mashup/gadget portal, open identity components, role-based access control security models, and open source content options.

Activities
The research team is working with MITRE's IT organization to deploy an open source, external partner-facing portal, plus content and service gadgets.  We plan to integrate existing multi-organizational collaboration capabilities (Handshake/Elgg and Sharpoint) into the common account management, the OpenID Connect identity protocols, and the other open standards in our integrating architecture.  Finally, we will explore the integratability of open source content capabilities such as WordPress and mediawiki for providing the content layers for collaboration and information sharing.

Impact
The power of being able to leverage not just the knowledge and resources within a single enterprise but across participating partner organizations is huge. Traditional barriers to cross-organizational collaboration have included security concerns, lack of open standards for supporting sharing across applications, identity and access control limitations, and the notion of an activity stream as a workable subscription delivery mechanism.  The commercial Internet and open source groups are slowly making the components available to solve this problem in the enterprise collaboration space.  We will demonstrate that it is possible to do this in a trusted way, allowing work forces across an ecosystem of organizations to work more efficiently by broadly leveraging related knowledge.  A related longitudinal evaluation research project will combine subjective and objective evaluation techniques to measure the business impact.

Public Release No: 13-0716

Problem
Improving efficiency at core airports is one of the primary goals of the Federal Aviation Administration. This research is targeted at improving efficiency through increased runway throughput. At airports where there is a dependency between arrivals and departures, a static interval between arrivals is commonly provided by Approach Controllers to accommodate the current and expected departure demand. The static spacing is often maintained even without waiting departures, negatively influencing runway throughput.

Objectives
This team believes that runway throughput can be improved by providing Approach Controllers with dynamic spacing guidance that accounts for the current and expected departure queue. To prove this, we built a prototype that provides automated arrival spacing guidance; it communicates arrival intervals depending on the type and order of departure aircraft queued at or taxiing to the dependent runway. It gives Approach Controllers an indication to use minimum arrival spacing when there are no scheduled departures.

Activities
We are now focusing on: 1) Providing guidance to the Tower Controller on the most efficient departure plan; 2) Improving the precision and stability of spacing guidance; and 3) Prioritizing arrivals or departures based upon demand; the automation provides spacing guidance that favors one over the other. We will also explore other possible applications for the concept, such as providing time-based spacing guidance on the final approach course.

Impact
In December 2012, we conducted fast-time and human-in-the-loop (HITL) simulations to assess the feasibility of this solution. Results showed that Controllers achieved a high level of conformance to guidance and workload levels were within a safe range. We found that both arrival and departure throughput can potentially be increased by using spacing guidance, as evidenced in the throughput data from HITL at airports where there is a dependency between arrivals and departures.

Public Release No: 13-0720

Problem
Critical medical information is buried in the free text of electronic medical records. Current medical extraction systems identify certain concepts well, but technology gaps limit their ability to represent meaning accurately. This can lead to errors in downstream applications (e.g., patient cohort selection) that use the extracted concepts.  For example, keyword search does not effectively capture uncertainty or negation, which may be critical in applications.

Objectives
The MITRE Assertion Status Tool for Interpreting Facts (MASTIF) is an open source module that builds on the functionality of an existing open source medical concept extraction system known as cTAKES (clinical Text Analysis and Knowledge Extraction System), which was developed by the Mayo Clinic. MASTIF extracts information to detect negation, uncertainty, and hypothetical contexts, allowing it to represent the “assertion” status of medical concepts.

Activities
We are adding features that will enable MASTIF to perform well on a wider range of clinical record types and that help increase accuracy. We have incorporated additional classifier components for more accurate representations of assertion, and we are training the system with new annotated corpora. We have extended our document structure analyzer to recognize new types of document structure. We have refactored the assertion engine in ways that will make make it easier to extend; simpler to retrain with different feature sets, parameters, and data sets; and easier to adapt to different Natural Language Processing pipelines. We plan to evaluate the accuracy of the enhanced system using available annotated data, and to evaluate its impact when incorporated into clinical applications used by the Department of Veterans Affairs researchers.

Impact
MASTIF currently determines assertion status for a variety of medical concepts, including medical problems, medications, and procedures, and has been integrated with cTAKES, which is now being distributed under an Apache license. This technology is addressing needs expressed by our sponsors, such as automatic patient cohort selection and clinical document retrieval.

Public Release No: 13-0190

Problem
MITRE sponsors with deployed personnel must maintain field-to-headquarters communication services independent of conventional infrastructure, which may be nonexistent, damaged, or denied. These services have traditionally consisted of independent radio systems with little redundancy, a minimum of remote control, and a high dependence on manual operation. Some users have embarked on building more flexible and adaptable radio systems. These, however, are for operation within independent systems, without autonomy or the open architecture necessary to leverage economies of scale for an Autonomous Radio Communications System (ARCS) to be  affordable.

Objectives
The idea is to increase radio transport service availability and reliability and reduce risk and operating costs by applying the technologies of modular, software-defined, cognitive radios and machine learning to an ARCS. The system would be able to sense the environment, reason, infer, and draw conclusions. Addressing this as a decision problem, we will use a configuration method to set selected transmission method parameters (e.g., waveform, compression) to maximize the probability of message delivery success.  This system will be designed to address significant constraints, such as minimizing the decision-relevant information stored, communicated, and processed at vulnerable locations. It will employ user-defined levels of automation. This work builds on MITRE's core stengths in machine learning, autonomous systems, and cognitive radios. The team's goals are to: -Produce a working prototype autonomous radio system that includes the algorithms an operational system would need -Create a prototype that will operate in a simplified, laboratory environment to  accurately represent the real, operational environment in which missions would be conducted -Identify the risks as well as the mitigation activities for making this prototype operational -Assist the sponsor in transitioning this effort to an operational status; and (4) -Further increase MITRE’s expertise in the core competency areas of machine learning, autonomous systems, and cognitive radio systems. 

Activities
During FY12, the team conducted research that identifies and describes the operational environment in which this system would need to operate.  We developed use cases, sequence diagrams, and other systems engineering analyses that explicate the characteristics of the requisite cognitive radio algorithm.  We also identified some potential target COTS and GOTS radio hardware and researched both spectrum and Internet sensing technologies and tools. This year we are developing a working prototype, as well as a roadmap for transitioning our technology to operational status, which will include identifying and recommending mitigation paths for critical risks.  

Impact
ARCS will operate more efficiently and effectively than the legacy system it would replace by providing increased message delivery success, using diverse paths and tailored transmission parameters, and by enabling capital resource consolidation and more efficient usage of scarce human expertise.

Public Release No: 13-0801

Problem
Detecting pathogens of concern within dirty environmental samples poses challenges for current detection methods (e.g., PCR and antibody technologies) due to chemical variability within the samples.  New applications are needed to expand the capability to detect pathogens that may be the result of a biological attack.

Objectives
Bacteriocin enzymes are produced by bacteria in nature as weapons in an effort to kill neighboring bacteria that may take vital resources from the producing bacterium.  In order to kill the target bacterium, the bacteriocin must attach itself to the target by binding to the cell wall or outer membrane of the organism.  The modular construction of these enzymes allows the separation of the binding activity from the killing activity.  Because these enzymes have naturally evolved to withstand variability in environmental conditions, the binding activity of the enzymes can be used to capture pathogens in dirty environmental samples where traditional detection methods will not work, leading to detection of pathogens of concern.

Activities
In FY13, the Bacteriocin Project will generate a journal manuscript describing the discovery and characterization of a novel protein antimicrobial, named Mitrecin A.  The patent application protecting the intellectual property of Mitrecin A will continue to be pursued by the MITRE Technology Transfer Office.  Experimentally, the Bacteriocin Project is working to characterize the second protein antimicrobial discovered in the program.  These experiments will be performed in collaboration with George Mason University.  Both enzymes will be evaluated as binding reagents in the development of a pathogen capture platform.

Impact
The Bacteriocin Project has fully characterized its first bacteriocin, Mitrecin A, as a novel enzyme.  A patent application protecting the intellectual property of the enzyme has been filed with the U.S. Patent and Trademark Office, and the results of the experiments surrounding the new enzyme were presented at the 2012 American Society for Microbiology General Conference.  In addition, a journal manuscript is being prepared for submission to publication.  The manuscript describes the discovery and characterization of the new bacteriolytic enzyme, which kills pathogens of national concern including Salmonella, Shigella, Aeromonas, Vibrio, and Yersinia species. Continued isolation and discovery of new bacteriolytic enzymes will provide a greater pool of pathogen capture options in the development of a capture platform.  Beyond the main goal of the Bacteriocin Project, the bacteriolytic enzymes have potential as biohazard containment agents, food preservatives, agricultural pathogen deterrents, and therapeutics against drug-resistant pathogens.

Public Release No: 13-0827

Problem
The complexity of the financial system makes it extremely difficult to aggregate, manage, and analyze the large proprietary data sets necessary to research potential threats to market stability. Traditional analytical and regulatory approaches have come under scrutiny due to their perceived failure to identify or prevent major financial disruptions. For example, the Flash Crash of 2010 and a series of mini-flash crashes that have been observed since then have undermined confidence in the capital markets.  Almost three years after the event, the regulatory community is still trying to find satisfactory answers to a number of related questions: -How did High Frequency Traders (HFT) and other categories of traders behave in the market on May 6, 2010, known as the Flash Crash? -What may have triggered the Flash Crash? -What role did the HFT play in the Flash Crash?

Objectives
The first question above is clearly an empirical question; however, the other questions are less empirical and lend themselves more to modeling. Our goal is to demonstrate that advanced computational capabilities (e.g., MITRE’s Infrastructure for Complex-Systems Engineering and new Agent Based Modeling techniques) can be applied to answer critical market stability questions such as those above. Given the publicly expressed interest in finding solutions from the Financial Stability Oversight Council and its member agencies (including the CFTC and SEC), as well as several major exchanges (e.g., NASDAQ, NYSE), there are a variety of partners available to help inform this research and potentially provide data.

Activities
MITRE is engaged in discussions with the regulatory community and the private sector, with the goal of validating the research plan and demonstrating our ability to securely house and analyze the big data sets necessary for this research.

Impact
Unlike previous analytical efforts, MITRE’s approach is both descriptive and generative, builds on prior empirical analysis of equity market data, and extends that analysis to futures markets.  In addition, we are not beholden to any theoretical perspective on how markets should behave but instead are free to characterize how real-world markets actually do behave. The potential impacts of this research include more effective capital market surveillance and analysis capabilities for regulatory agencies, as a result of improved understanding of HFT and Flash Crashes.

Public Release No: 13-0555

Problem
Pathogens can cause disease outbreaks with devastating consequences, including loss of life, societal disruption, and large economic costs. For national security, law enforcement, and public health purposes, it is important to determine where an outbreak pathogen came from and whether the outbreak was natural, accidental, or intentionally caused. Recent advances in biotechnology and bioinformatics can be leveraged to answer these questions, but methods are needed for applying these to attribution problems. 

Objectives
We are designing tools for managing and integrating data from multiple laboratory techniques and information sources to support attribution activities. We are also developing new methods for determining whether a pathogen was produced in a laboratory versus from a natural source. This includes investigating chemical signatures indicative of growth in lab culture and DNA sequence signatures indicative of a history of laboratory culture.

Activities
We are building data integration tools using LabKey, a platform for biological data. These tools include capture of attribution metadata, automated import from key public information sources, import of laboratory results, and export for investigation analyses.  We are also engaging with other researchers to integrate additional biological methods and develop a collaborative platform. In the laboratory, we are analyzing Volatile Organic Chemicals (VOCs) produced by bacteria.  Finally, we are developing bioinformatics methods to identify DNA signatures indicative of lab culture using whole genome sequence data and phylogenies.

Impact
We have created a database for capturing metadata with automated import from a few key information sources. With further development and the addition of data types from other researchers, these data tools will allow investigators to more rapidly and comprehensively discover and evaluate potential sources of an outbreak pathogen. DNA signatures of lab culture could provide a new capability by distinguishing pathogens from natural sources from those released from laboratories in patient and field samples.   

Public Release No: 13-0348

Problem
Surface surveillance can enhance controllers’ situational awareness at an airport and improve the safety of operations. Existing surveillance systems use a combination of advanced capabilities (e.g., multilateration) and are therefore cost prohibitive for small airports.  This has left more than 450 U.S. towered airports without an affordable surveillance solution.

Objectives
A block occupancy-based surveillance concept and prototype can be enabled with the use of inexpensive magnetic sensors. Under this concept, the surface is divided into operationally relevant blocks, and magnetic sensors are used at block boundaries to monitor aircraft and vehicles entering and exiting these blocks. As targets move on the surface, the sensor detects movement in or out of a block and the occupancy status of each block is displayed in the tower.

Activities
In previous work this research validated the operational acceptability of this concept through human-in-the-loop simulations.  Current research is focused on developing a simulation of aircraft-generated sensor data and algorithms to integrate sensor signals to determine block occupancy. The sensor simulation will be derived from, and the block occupancy algorithm design will be validated through, tests conducted with magnetic sensors.

Impact
The system is expected to improve surface safety at small and medium-sized airports where advanced surveillance capabilities are not affordable. The proposed system may also serve to support airport operations when visibility is low, to reduce the impact of field-of-view issues from the tower, and/or to improve situational awareness of the airport surface for non-towered airports.

Public Release No: 13-0752

Problem
As the Federal Aviation Administration (FAA) evaluates evolving procedures and technologies for NextGen, an objective, real-time, and continuous measure of workload can be critical to assessing impact on pilot and controller performance. EEG measures provide a basis for metrics of cognitive workload, vigilance, and attention. Recent advances in EEG technology permit better data quality and system usability.

Objectives
This research team is developing a real-time EEG monitoring capability that can be used during Human-in-the-Loop Simulations (HITLs) to assess NextGen automation concepts in terms of operator workload and effects on performance.

Activities
Results of our experiments to date indicate a high correlation of EEG with task load levels in working memory tasks and moderate correlation with Air Traffic Control tasks.  This year’s research is focused on building the real-time data analysis and visualization capabilities, and validating the workload metrics in the context of a laboratory HITL.

Impact
Our work will provide an objective real-time workload metric to facilitate FAA decisions about introducing candidate technologies that could potentially impact human performance. In addition, the BrainGage tool provides a general cognitive monitoring capability for other applications in human factors/neuroscience experimentation, adaptive aiding, advanced human-computer interaction design, and cognitive training.

Public Release No: 13-0684

Problem
Commonly used virus detection and characterization systems rely on specialized “hard-coded” (and expensive) techniques to detect known gene sequences or anticipated virus surface proteins.  Emergent or synthetically engineered viruses often elude detection by such methods, suggesting a more general and cost-effective approach is required.  Recently developed knowledge regarding the cellular mechanisms of viral replication and biology enable the detection of previously unknown viruses through the use of fluorescent labels combined with cell-based technology.

Objectives
Recent advances in understanding the roles of proteins engaged in viral replication enables researchers to detect entire families of viruses with one easy to use method.  Our research focuses on applying this approach to the dengue family initially, expanding later to other priority pathogens while focusing on detection speed, sensitivity, and breadth of coverage.

Activities
The key to developing a broad-based detection mechanism lies in the creation of artificial RNA constructs that are triggered after viral infection. RNA construct-encoded genes yield fluorescent activity when incoming virus replication starts.  Success with dengue viruses will enable the effort to broaden research into the detection of other viral families of interest such as ebola virus and alphaviruses.  Results from this program have been presented at multiple MITRE research program events and at the following forum during 2012: UPR School of Medicine, Graduate School Biomedical Sciences Seminar Series.  We are transitioning the technology to Oak Ridge National Labs and the Centers for Disease Control (CDC).  The CDC has established a working protocol and is assessing limits of detection for the dengue family of virus.

Impact
We are developing a cell-based technology for broad-spectrum detection of viruses.  Our research concentrates on detection of unknown, emerging, and genetically altered viruses and complements existing genome amplification-based technologies. This cost-effective approach may become a first-tier sensing countermeasure of interest to national security and public safety. 

Public Release No: 13-0688

Problem
Various organizations need ways to miniaturize sensing device size to chip scale and deliver high performance, i.e., high sensitivity and low false alarm rates, to close a critical gap in trace explosives detection. For an operationally effective technology solution,  the key requirements are reliability of performance, ease of operation, and adaptability for rapid integration with other sensing modalities to accurately determine continuously evolving threat signatures.

Objectives
Ion mobility spectrometry (IMS) is the most widely used trace-detection technology. A significant leap in the IMS technology is feasible through a paradigm shift in the selection of the drift medium. A condensed-phase medium can drastically miniaturize IMS and enhance the resolving power (reducing false alarm rates). With optimal operating conditions a condensed-phase IMS device can yield resolving power high enough to differentiate the interferences in a traditional gas-phase IMS and reduce false alarm rates.

Activities
Our FY13 activities include: -Enhance modeling and simulation capabilities to understand the science of effective separation -Design IMS and control space-charge to achieve high-resolving power -Demonstrate reliable and effective separation of analytes in condensed-phase IMS -Build design and fabrication steps for a chip-scale condensed-phase IMS.

Impact
We have demonstrated proofs of analyte ionization, ion injection, transport and detection, and published these results in "Analytical Chemistry," vol. 84, no. 21, pp. 9295-9302, 2012. We continue to focus on proof-of-ion separation and reliability assessment for a wide range of analytes relevant to military, law enforcement, and first responder organizations. We are also strengthening MITRE’s leadership in novel sensing and acquiring signatures through broader integration with other modalities, which will benefit our sponsors. 

Public Release No: 13-0619

Problem
Malware authors use cryptography to hide malicious code, evade signature-based security tools, and conceal network traffic. Against such malware, security teams are hard-pressed to detect persistent infections and stolen data leaving their network. The current defense against malware cryptography is constant scanners, which search executable files for strings that indicate use of cryptographic functions. Unfortunately, these scanners are trivially bypassed by modern malware builders and cannot detect cryptography in a generic fashion. Heuristic methods of cryptography detection based on statistical properties of code have been explored in academia, but require further study  and  refinement  to  be  effective  in  operational  scenarios.  Further,  existing heuristics largely focus on detecting block ciphers and hash functions and barely address stream ciphers or asymmetric encryption.

Objectives
CIPHOS aims to create robust classification heuristics capable of locating cryptographic functions in executable binaries using statistical analysis. These classification tools use statistics of a function’s operations and flow structure to differentiate between cryptographic and non-cryptographic functions. Three classifiers were constructed from a ground truth data set using machine learning techniques; neural network, decision tree, and naïve Bayes. CIPHOS classifiers can be incorporated into static analysis tools, such as IDA Pro, memory analysis tools such as Volatility, and dynamic analysis frameworks.

Activities
Two Python prototypes were developed during FY12: a standalone tool for static binary analysis and a plugin for the Volatility memory analysis framework.

Impact
Preliminary results on data gathered from Windows libraries, applications, and malware samples show promise for operational use. The classifiers will be refined and tested against a larger dataset in FY13.

Public Release No: 12-5137

Problem
In satellite communications (SATCOM) systems, scenarios in which the line-of-sight link between a ground terminal and the satellite is fully or partially blocked can occur often. This is especially true in urban environments, for example, in cases where mobile SATCOM terminals pass by structures that obstruct the view to the satellite being used for communications. This is known as hard blockage.  In other scenarios, the link between the terminal and the satellite might be partially blocked because the transmitted signal has to propagate through heavy foliage, and, as a consequence, suffers intense attenuation, especially at Extremely High Frequencies (EHF). This is called soft-blockage. Similar blocking situations occur in terrestrial communication systems, but we are focusing only on SATCOM applications, in which there are many users that encounter soft or hard blockage situations, which result in severe impact to their connectivity, throughput, and theater operations. Some of the solutions that are presently being sought by users of tactical SATCOM systems include the use of coding with extremely long interleavers, the use of complex Automatic Repeat Request (ARQ), or the use of multiple satellites. However, all these approaches either do not work well or tend to be complex and expensive, requiring considerable hardware and planning. 

Objectives
Our plan is to design and deliver a low-complexity blockage-mitigating solution for EHF SATCOM based on the use of Fountain codes to solve the hard or soft blockage problem with a focus on mobile ground terminals. This approach can also be used for covert operations to effectively ensure that the user receives the transmitted data with very high probability and without requiring acknowledgements or requests for retransmissions.

Activities
When Fountain coding is used, the receiving terminal simply continues to collect the packets of fountain encoded symbols that do get through the blockage, as the channel becomes unblocked (for some period of time, only to become blocked again, on an intermittent and random basis) until it has the necessary set of encoded symbols to decode the entire message. There are very powerful Fountain codes with efficient encoding and decoding algorithms that allow the recovery of the original k source symbols in the message from any k' encoded symbols with high probability, where k' is just slightly larger than k. In this project, we plan to use these Fountain codes, which are close to optimal. We will develop SATCOM blockage models for both hard and soft blockage, which are consistent with empirical data, and implement them in software. We will then select appropriate Fountain codes for data transmission over SATCOM channels with soft or hard blockage and will develop comprehensive computer simulations of the system operation to obtain assessments of performance in soft and hard blockage. Some of the information that we will obtain will include a characterization of performance in terms of message error rate as a function of signal-to-noise ratio (Eb/N0) as well as statistics on the average time for completion of message transmission, the achievable throughput, and estimates of channel capacity. The methodology that we will use in all the assessments of performance will be based on a combination of analysis and comprehensive computer simulations.

Impact
We will provide an effective, low-complexity and low-cost solution to the soft or hard blockage problem in protected SATCOM systems. This solution will improve the overall system capacity and the terminal’s connectivity and data throughput, especially for Comm-on-the-Move terminals, which need to operate in environments where link blockage can be expected to be present in various degrees for a considerable portion of the time. 

Public Release No: 13-0330

Problem
Communication between an Unmanned Aircraft System (UAS) and an Air Navigation Service Provider (ANSP) in non-segregated international oceanic airspace currently employs a telephone land line to the ANSP facility supervisor or ARINC or SITA. The telephone voice communication process is cumbersome and increases delays, errors, and workload for all involved. This approach does not scale to the projected future demands of UASs flying in non-segregated international oceanic airspace.

Objectives
Our idea is to create a Controller-Pilot Data Link Communications (CPDLC) emulation for use by UASs that lack Future Air Navigation System 1/A communication.  Our research goal is to enable efficient communication using the existing data link infrastructure between UASs and the ANSPs responsible for providing service in an International Flight Information Region.  Results will be demonstrated in a field lab environment.

Activities
We have identified an existing UAS, Broad Area Maritime Surveillance-Demonstrator (BAMS-D) for use in demonstrating this research. We have developed the concept of operations, software architecture, and prototype integration approach through interactions with BAMS-D program managers.  Next we will complete architecture and integration plans and develop the software-based emulation environment, followed by an operational demonstration and validation of the benefits analysis.  Additional activities will include simulated mission planning, coordination with the Federal Aviation Administration, and demonstration planning. 

Impact
The impact of allowing UASs to integrate into international oceanic airspace worldwide using the existing Controller-Pilot Data Link network is significant. Our work will also benefit many other organizations. For example, the Department of Defense projects that many of its 6th generation aircraft will be unmanned, and this research will help quantify the operational benefits of implementing CPDLC in these aircraft.  Our work will also be of value to the other civilian government agencies that are operating large UASs in non-segregated international oceanic airspace. 

Public Release No: 13-0618

Problem
Given that Unmanned Aircraft Systems (UASs) have no onboard pilot to perform “see and avoid” duties, an automatic (i.e., empowered to act without pilot involvement) mechanism is needed to enable UASs to avoid conflicts with other aircraft when the command and control link to the pilot is not available.

Objectives
Through a progressive series of experiments and flight demonstrations, this research team will continue to explore the technical and operational issues associated with using NextGen Automatic Dependent Surveillance–Broadcast (ADS-B) technology as a surveillance source for onboard, automatic sense-and-avoid mechanisms.

Activities
Using a MITRE-developed computer simulation (algorithmEvaluator) and flight testing, MITRE—in collaboration with investigators from NASA Langley Research Center, the University of North Dakota, and Draper Laboratories—will evaluate the technical and operational feasibility of cooperative automatic “sense and avoid” alternatives and support ongoing validation efforts for analysis tools and infrastructure.

Impact
Through the algorithm development efforts, computer simulations, and flight evaluations,  we will explore technical and operational issues and generate data that can help inform policy decisions in the aviation community, expand “sense and avoid” concepts and architectures, and accelerate the development of the appropriate technology and standards to enable the integration of UASs into civil airspace.

Public Release No: 13-0727

Problem
There has been little systematic research and analysis done on the concepts of denial, deception, and counter-deception in the cyber-space domain. Classical deception theory, however, has been comprehensively researched and could be applied to the cyber domain.

Objectives
We will systematically apply deception theory and research to cyber security to provide unique insights and approaches to cyber defense, cyber situational awareness, and cyber intelligence. We will also develop foundational products to inform network defenders and security researchers and transition successful technology to the government.

Activities
We are leveraging MITRE resources for modeling, simulation, and experimentation--and collaborating with other cyber researchers and practitioners--as we validate our concepts and prototypes. Our foundational products include: -Taxonomy: A terminology bridge between cyber security and the traditional deception domain tactics, techniques, and procedures. -Capability Maturity Model: Descriptions of deception techniques in a framework that allows extraction of observable indicators of deception operations. Network defenders can then assess their own deception abilities, as well as those of an adversary. -Integration of the cyber kill chain and deception: Use of the cyber kill chain framework to describe concepts of operations in how deception tactics could be deployed against an advanced persistent threat. -Use case scenarios: Description and analysis of cyber-attack scenarios, with both sides using deception on offense and defense. These basic scenarios examine the benefits, challenges, and implications of deception for defensive purposes. -Deception whitepaper on our work, which will be submitted for publication in a peer-reviewed computer security journal.  

Impact
This research will help cyber defenders use deception tactics to thwart advanced persistent threats by forcing adversaries to move more slowly, spend more resources, and take bigger risks, and by misleading adversaries into compromising their stronger tactics and relying more on their weaker tactics.

Public Release No: 12-5008

Problem
Graphene is an allotrope of carbon in which the atoms form a single atomic sheet arranged in a two-dimensional honeycomb lattice. It has novel mechanical, optical, electronic, and chemical properties with important potential applications in many areas of technology. The rapid pace of applied research combined with advances in graphene synthesis technology suggest that graphene-based devices will be of great practical importance in the near term. Promising applications include optical modulators, terahertz devices, chemical sensors, and nanoscale electronics. Given the current high level of interest in graphene, it is important to develop an understanding of where graphene is likely to entail breakthroughs in technology and the potential effect of those breakthroughs on practical areas of interest. 

Objectives
Given the broad spectrum of research on applications of graphene, our initial investigation concentrated on review articles that summarize the current state of graphene research.  We also undertook to understand the physical basis for graphene’s novel properties and to gain familiarity with fabrication techniques that will influence the availability and quality of graphene samples for applications of interest.

Activities
We have completed our initial assessment of the scope of graphene research with emphasis on the broad spectrum of potential applications.  We have identified a half dozen application areas that are likely to be of practical interest to our sponsors and show significant benefits within the decade. We will document our findings and identify two or three areas for more in-depth investigation in FY14. 

Impact
Through our research, we have identified 42 potential applications that utilize several unique features of graphene, including its extremely high tensile strength, electrical conductivity, thermal conductivity, and optical absorption, as well as its flat optical response across a broad spectral range.  Three applications have been selected for immediate further investigation: broad spectrum mode locking, fast miniaturized optical modulators, and electromagnetic interference shielding.  We have also identified a second tier of three applications: THz generation, high-speed photodetection, and hypersensitive gas detection.  The second tier applications may be subject to change as our understanding improves.  Our intent is to follow up this study next year with more detailed investigations of two or three areas in collaboration with researchers both within MITRE and in academia.  This study will greatly improve MITRE’s ability to advise our sponosrs on the potential impact of graphene on technologies of interest and the best areas for strategic investment in non-commercial research. 

Public Release No: 13-0522

Problem
In modern computer networks, the burgeoning flows of email, social media, and application content are gathered into datasets that contain huge quantities of communication events describing dynamic networks. A key challenge to understanding these networks’ behavior lies in predicting how their structure will evolve--that is, how new links will appear and old links will fade. The applications of such a link-prediction capability are diverse.  They include inferring unobserved or soon-to-be-formed links in criminal, terrorist, and insider-threat communities. They might also be used to assess how unusual a current behavioral change within a computer network is. The problem with link-prediction today is that the available techniques do not scale in size and speed well enough to allow accurate and timely analysis; do not incorporate content information; and do not run on existing or near-term planned sponsor infrastructure.

Objectives
We will provide a highly scalable set of algorithms that perform link prediction at higher accuracies than are achieved currently at similar scales.  We will pursue and extend recently emerging techniques to develop graph algorithms that can combine the analysis of a network’s structure over time with analysis of the content flowing within that network. These algorithms are expected to work and scale on existing and emerging cloud infrastructure and platforms.

Activities
We will focus on four major modes of cross-cueing the network, content, and geospatial information.  These modes will be combined from the four core capabilities of link prediction, community detection, edge clustering, and entity resolution. To evaluate our methods, we will focus on real datasets, starting with the Enron dataset and one (or more) additional large dataset that allows us to highlight various forms of cross-cueing. The four modes of cross-cueing for link prediction that we intend to evaluate are: a) the baseline, where we run standard link prediction from social measures, but without community detection features, b) where community detection information is used as a feature, c) where cross-document coreference information of named entities is used to assess content similarity and thus content-augmented social network measures, and d) where community detection is accomplished by clustering events along edge models.

Impact
Our research will demonstrate a way to better identify bad actors in social and cyber networks and monitor their current and predicted activities more effectively within very large datasets. The end result will give our civilian regulatory, law enforcement, and military organizations capabilities that provide prescient insights into the suspect behavior of individuals and groups. The automated prediction of impending links for the communities in which they are embedded will be based on both the content and metadata passing across the networks in volume. This will enable our sponsors to integrate more of their data while pursuing social network analysis of individuals and groups in a manner that will provide for accurate and timely analysis, even for very large datasets.  We expect our approach will result in a highly productive analytic development platform that works on already scalable cloud systems, some of which are already deployed. This means that the response time to address new problem sets as they arise should be significantly reduced, which will increase the agility of our sponsors’ mission enterprises.

Public Release No: 12-4626

Problem
Surveillance systems span numerous sensor modalities but most prevalent and accessible are RADAR and optical sensors.  Intelligence analysts and decision makers look across these sensors to derive some new understanding of “who, what, when and where,” ultimately trying to understand and track enemy capabilities, organizations, motivations and intentions.  This information is then used to inform decisions on mission plans, counter measures, capabilities development, training, and more.  However, when detections are successfully identified or characterized, it is difficult to associate and link this information across RADAR, Wide Area Motion Imagery, and Full Motion Video (FMV) in the midst of jitter and misalignments in space and time.  If the measurements of objects gathered across surveillance systems could be sufficiently correlated, then questions of “who, what, when and where” could be better answered and a higher-level of understanding could be achieved.

Objectives
We will research and combine emerging algorithms in geometric computer vision and airborne radar to achieve improved calibration, modeling accuracy, and correlation.

Activities
Many platforms contain multiple sensing modalities.  In the first stage of this project, we will develop new correlation and calibration techniques for combining information from multiple sensors that are located on the same platform. In stage 2 of our research, we will extend our developments to multiple platforms that are observing overlapping regions.

Impact
We have developed a new approach for improving the global pointing accuracy of an airborne phased array radar system by 7x by adding a bore-sighted FMV sensor running a Structure from Motion computer vision algorithm. We are currently studying the effects of pointing accuracy improvements on downstream RADAR algorithms.

Public Release No: 13-0592

Problem
Today, intelligence analysts have no formal representation of their consumers' decision spaces that could enable analysts to better assess what information is critical to consumers' decision making. In addition, without the use of decision spaces, members of the Intelligence Community (IC) are currently unable to adequately share their understanding of plausible future world states and the impacts of operational actions on those states. This project is both the umbrella project and one facet of the multifaceted Crystal project. A graphical depiction of a decision space provides a visual summary of options, their outcomes, and the factors that affect those outcomes. In past research using decision spaces for emergency response, we have shown that this visual synthesis yields better, quicker, and more confident decisions. This approach can also provide a common orientation that facilitates collaboration among decision makers. The other facets of the Crystal project address decision spaces for Healthcare, Communication audiences, and Command and Control decision makers.

Objectives
We will elicit formal representations of decision spaces using the cultural network analysis method, combining interviews with a small cohort and computer automated surveys with the greater IC. These decision spaces will provide a visual summary of options, their outcomes, and the factors that affect those outcomes, and therefore can provide a referential artifact to establish common ground among the IC and with intelligence consumers. In this way, we can use decision spaces to enable members of the intelligence analysis community to share their understanding of how factors in the operational environment will impact operational outcomes.

Activities
We will then design the decision space collection and collaboration environment. Beyond gathering the consensus of the analyst community around a particular hypotheses or outcome, the environment will also enable us to gather causal models of the reasoning behind those opinions. The prototype system will be evaluated with a panel of MITRE analysts.

Impact
This work will enable intelligence analysts to develop a model of their understanding of how factors in the operational environment will impact operational outcomes through collaborative decision spaces, share that understanding with other members of the intelligence analysis community, and thereby enable joint choices that will achieve good outcomes across the broadest swath of plausible future conditions. This prototype will improve on current tools by focusing on forecasting the plausible states of a target after consumer options are executed; consequently it will more effectively convey a more accurate model of actions, the values they achieve, and the rationale for doing so. Enabling this new focus will facilitate effective intelligence analysis collaboration by providing analysts with the ability to collaboratively share evidence, develop structured arguments, debate judgements, and improve their ability to reach consensus on intelligence issues.

Public Release No: 13-0623

Problem
The SWaP (size, weight, and power) of next-generation wireless electronic systems for the warfighter is a concern for many of MITRE’s sponsors. Whether the warfighter is dismounted, in a vehicle, on a seagoing vessel, or in an aircraft, SWaP is always a primary limitation on the system. If a wireless electronic system can provide the needed capability with less SWaP then it leaves room for additional capabilities needed by the platform. Wireless electronics can be viewed as two subsystems: RF/analog and digital. Reducing the SWaP of the digital components can be done by taking advantage of advances in technology, such as using a next generation field-programmable gate array or a custom digital application-specific integrated circuit using the latest semiconductor processes. Reducing the SWaP of the RF/analog section historically has been done with custom RF integrated circuits (RFIC) or system in a package (SiP) techniques. RFIC development can be costly and time consuming, which makes it prohibitive for most companies and not feasible for prototyping or low quantity applications. MITRE has a number of hands-on efforts to produce reduced SWaP prototypes, but does not have much experience using SiP design. However, we can learn from and leverage the expertise of companies that specialize in SiP integration.

Objectives
Our goal is to improve MITRE’s capability to develop and deliver highly integrated, SWaP-constrained wireless prototypes by adding SiP design techniques to our toolbox of design capabilities. This will be accomplished by identifying key commercial players in this space and working with them to gain knowledge about the capabilities of their SiP technology. Ultimately, we will implement one of MITRE’s RF/analog subsystems as a SiP to demonstrate this capability.

Activities
We have identified three SiP vendors that have been accredited as Trusted Suppliers in the Trusted Foundry program. MITRE’s engagement and investigation of these vendors' capabilities is in varying stages. One has already provided a ROM to implement an existing RF/analog design of MITRE’s as a SiP. A second company is working on an equivalent ROM, and the third is still in the process of putting an NDA in place. From the first company we’ve learned the design tradeoffs necessary to turn an existing design into a SiP using their process. Once we have information from at least one more company, we will compare their capabilities and downselect. The next step will be completing the detailed engineering design work necessary to produce a SiP prototype in partnership with the vendor.

Impact
The ability to use SiP as part of our RF/analog designs will greatly improve MITRE’s capability to deliver highly integrated, reduced SWaP RF/analog front-end prototypes. This knowledge will complement our custom digital ASIC flow and FPGA design capabilities. The understanding and practical, hands-on expertise with SiP technology will also allow MITRE to provide technical advice to government programs seeking reduced SWaP.

Public Release No: 13-0302

Problem
Resources are limited when it comes to raising mission assurance in the face of increasingly significant cyber campaigns and activities by the advanced persistent threat. Decision-makers need a better understanding of the effects of their architectural and operational actions on cyber adversaries.

Objectives
While metrics are desirable decision aids, because of data sparsity, alternative approaches to identifying and evaluating evidence are also needed. Our goal is to identify and characterize analysis techniques so that evidence can be gathered and metrics evaluated in rigorous, meaningful, and operationally feasible ways.

Activities
We will provide an initial common vocabulary for effects on cyber adversaries to facilitate identification of relevant evidence. We will identify and characterize analysis techniques and define a representative set of evidence and metrics. Then we will perform a pilot or case study applying at least one analysis technique and set of evidence/metrics, and provide initial guidance on the  effective use of analysis techniques and metrics.

Impact
Our work will enable more meaningful and useful comparison of alternatives based on their effects on cyber adversaries.

Public Release No: 12-5124

Problem
In complex, data-rich environments, customers (e.g., intelligence analysts, task managers, and planners) often need to investigate the ways in which information is derived and used.  Without being able to trace the source and derivation history of data, it is difficult for an analyst to build a level of trust that the information is appropriate for their use. Data provenance (sometimes called “pedigree”) represents a fundamentally new source of information about the sources of data and actual analysis workflows that previously has not been available. Provenance can tell customers where a data item came from, who used it, and how it was processed. Data provenance helps users make trust decisions in an environment that contains many different available sources of information.

Objectives
MITRE has developed groundbreaking data provenance capture and management techniques and software. In this research project, we will integrate our provenance capture and exploitation infrastructure with the Ozone Widget Framework (OWF). As the OWF is being used across the Intelligence and Defense communities, the results of this project will apply to many government agencies. This project will result in an evaluation system that helps analysts and managers assess the utility of intelligence information.

Activities
Our primary activity is to create a patch to the OWF to enable provenance collection, and then we will deliver this patch to an intelligence sponsor testbed. We will document how to set up, install, and use provenance collection. We also plan to create an OWF “Data Fitness Widget,” which will permit OWF users to see and use available provenance data. Once available, the OWF patch may be applied to many other sponsor environments as well, greatly broadening the impact of this work by taking advantage of the Government Open Source Software terms under which OWF is available.

Impact
Provenance information will help determine the Return on Investment for data collection systems and can reduce the time analysts need to determine whether a data product is suitable for their specific needs, by reducing their need to track down sources via phone or email. Provenance also enables users to flag anomalous workflows and potential security breaches, and it supports mentoring in which new analysts (or analysts working in new areas) can see how master analysts executed their missions.

Public Release No: 13-0675

Problem
One of the key goals of a command and control (C2) system is to enable commanders to make sound and timely decisions. Achieving this goal is a challenge when situations and outcomes are uncertain and decisions must be made quickly. The Decision Weaver project is focused on supporting agile and adaptive C2 by helping to manage uncertainty at the level of the judgments made and choices considered by decision makers.

Objectives
The goal of this project is to develop decision-support capabilities that can be automatically  assembled to help manage uncertainty for C2 decisions in a context-aware manner as efficiently as possible. The idea is that by dynamically providing customized decision support for specific decision tasks,  decision makers will be empowered to quickly manage uncertain situations and outcomes even when faced with new or unexpected problems.

Activities
We will design a collection of Bayesian network templates or fragments that summarize how uncertainty in a particular type of information or information source will affect a given type of time-critical decision. Given a set of relevant network fragments, the next step is finding a way to instantiate them dynamically as a customized chunk of reasoning in a given decision context, and then “weave” the pieces together into a coherent decision-support capability that can reason about the problem at hand.

Impact
If successful, this research will help give decision makers the support they need to find creative and adaptive solutions to new and challenging problems,  even in the face of incomplete and uncertain information. Decision makers will be able to make expedient decisions based on the best knowledge and information available, with the benefit of being able to assess the level of confidence they should have in those decisions.

Public Release No: 13-0437

Problem
Command and Control (C2) is changing: “military forces must continually transform and adjust to remain highly effective in extremely fluid environments.” New battlegrounds and unconventional warfare increases the reliance on rapid human decision-making processes. It is not possible to create a template for every possible decision that a unit may encounter ahead of time. Units on the ground must make expedient decisions, often in the presence of incomplete information about the situation. Decisions require experience and creativity, but without an understanding of how similar decisions were made, the material differences in situations leading to decisions, and an ability to capture the decision making processes themselves, it is difficult to learn from collective experience. Furthermore, knowledge of relevant decisions helps decision makers build greater confidence in the courses they choose. These requirements are key factors in the Assistant Secretary of Defense for Research & Engineering's Science & Technology priority called Data to Decisions (D2D). We propose a flexible, intelligent system (Denim) to support decision makers in crafting creative solutions to new and changing challenges, even in the face of incomplete information.

Objectives
Our overarching goal is to enable better human decisions in less time. We would provide decision makers with a decision-support system that notes and procures relevant information, based on past and suggested relevant workflows. The workflow model will be sufficiently general to be compared and modified based on changing needs. From a capability perspective, a part of this system would be a search interface on top of a repository of past decision-making processes and their outcomes. This repository would be a tremendous resource for applications that focus on improving key parts of decision-making. The research aspect of this outcome focuses on permitting searches for relevant workflows; the search problem here likely is not as simple as well-known text searching. These capabilities are an important baseline, because any application that seeks to improve the way a commander does (for example) resource allocation must have some background about how a variety of similar decisions were made in the past, in combination with a measure of the quality of their outcomes. Denim stores and manages that background and aids in the identification of what is “similar.”

Activities
Denim will focus on a few key subdomains of interest in C2, including Personnel Recovery, and we will begin by building a model describing the types of decisions that are necessary for the domain. By examining any examples of knowledge from the domain that are made available, Denim will develop methods of abstracting workflows at multiple levels; for example, very specific workflows can be abstracted up to more general versions of the same, sacrificing detail in exchange for generality. Very general workflows in turn may contain links to multiple different versions that apply to more specialized situations. By storing and retrieving workflows at several levels of generality, Denim seeks to find a balance point between general workflows that are widely applicable, but not very specific, and very specific but narrowly applicable workflows.

Impact
If we are successful, decision makers will be able to make crucial decisions faster, backed by a much broader base of organizational knowledge and experience.  The decision-maker's personal abilities will be augmented by more experience than any one individual may possess.    

Public Release No: 13-0138

Problem
In time diversity wireless and satellite communication, multiple copies of the same signal are transmitted during different time intervals to improve the reliability of reconstructing the transmitted message at the receiver. This is helpful in combatting deep fading. If the communication system is frequency hopped for protection against jamming and/or interception, the carrier frequency repeatedly changes after short dwell time intervals, which renders coherent receiver design infeasible. To overcome this problem, the system is designed to transmit reference signals of known symbol values to help the receiver with phase synchronization. In the traditional approach, the receiver uses only the received reference signals (which are corrupted by noise) for phase synchronization and demodulation. Multiple copies of the signals are independently demodulated before diversity-combining. Unfortunately, this approach allows for a bit at a particular position to be decided as 0 in one copy and 1 in another copy before combining. This manifests as a degradation in the bit error rate performance of the receiver. Time diversity is usually resorted to in order to deal with extremely poor signal-to-noise-ratio channels, and any degradation due to the inherent inability for phase acquisition should be minimized.  The problem is how to develop and implement a jointly maximum likelihood signal combining scheme when the signal phases are unknown and given that they can be different.

Objectives
Our idea is to formulate the maximum likelihood signal combining problem with the constraint that the symbol assignment for signals at corresponding positions in the multiple copies be identical. We have investigated this problem, developed optimal and nearly optimal solutions, and demonstrated improvements of from .5 to 2 dB over the above traditional method, for different system configurations. The team's goals are to implement variations and refinements of our signal combining algorithm in Field Programmable Gate Array (FPGA) hardware and benchmark their detection performance and speeds of operation. As part of this effort, we plan to produce functioning prototype hardware, FPGA source code, a report documenting the algorithms and results, an Intellectual Property (IP) disclosure for filing a patent, and a plan for technology transition.

Activities
We are experimenting with porting our algorithms to FPGA for implementation on our current FPGA hardware suite. This will help us select our final algorithms and, if necessary, an optimal target FPGA board. We will implement and test the algorithms, stressing them to the limits of their design specifications in terms of the size of the data segments, modulation types, number of copies of data segments, and data rates. We will document all our test method, detection performance, and throughput for various cases.

Impact
Successful execution of this task will provide the MILSATCOM community with a technically excellent solution for the critical future protected tactical waveform. It will also create a government-owned reference implementation of signal detection in frequency hopped systems operating with low rate repeat codes, which will in turn serve the public and advance the state of the art in diversity data communication.

Public Release No: 12-4657

Problem
The Smart Grid promises to provide power at greater efficiencies and lower cost; however, it requires greater coupling of power and communication networks. Some prior research, using just topological analysis, has asserted that this increased coupling may lead to a greater risk of cascading blackouts. Accounting for physical power flow as well as different coupling methods promises to yield far more insight into this risk.

Objectives
Combining complex systems analysis and power flow based cascading failure risk will move the discussion beyond toy models to focus on what the real vulnerabilities are.

Activities
We will produce a realistic and tractable model of the coupled networks comprising the Smart Grid (power and communications). We will also publish a peer-reviewed paper that describes and justifies this model as well as the different failure mechanisms in SCADA networks, Internet, and power grids.

Impact
Our work could: disambiguate the many aspects of the Smart Grid to refine policy discussions; counter prevailing findings from toy models by providing an engineering foundation for Smart Grid analysis; and identify risks for cascading blackouts in Smart Grid power systems, including components which introduce additional risk.

Public Release No: 13-0553

Problem
About 60 percent of all tax returns are filed by paid preparers. GAO audits have found that preparers make significant errors in preparing tax returns, and their actions significantly affect the Internal Revenue Service’s (IRS) ability to administer tax laws [GAO-11-336]. Recently, the IRS implemented a registration requirement for paid preparers that requires them to adhere to standards of practice. However each tax season, the IRS is faced with the challenge of detecting non-compliant or fraudulent preparers for proper treatment, ideally before issuing their tax refunds. Our project's objective is to help with this selection process.

Objectives
Our goal to develop a data analytics method for analyzing tax returns prepared by paid practitioners to detect non-compliant preparers. The detected areas are then used in three fashions: -Identify tax preparers who have committed non-compliance and/or fraud -Compare non-compliances over years and determine the trends -Predict evolution of fraudulent schemes by paid preparers and identify areas where future fraud is anticipated.

Activities
In FY12, we focused on unsupervised techniques to find patterns of anomaly in tax returns filed by paid preparers. These techniques are particularly useful when there is no data on which returns or preparers are fraudulent. In FY13, we are applying supervised techniques, using data on indicted tax preparers, to model non-compliance. Specifically, we are concentrating on two types of supervised techniques: 1) contrasting the known fraudulent cases with the rest of the population and learning distinctive attributes that characterize each population; 2) developing a predictive model, using the known fraudulent examples. We will then run the results by MITRE and IRS experts to take advantage of the domain knowledge and finetune our method.

Impact
Successful implementation of our methodology will enable detection of tax preparers who file fraudulent tax returns and help to improve tax compliance rates.

Public Release No: 13-0610

Problem
Obtaining information on the history or provenance of discovered objects (where an item was prior to its discovery) is of critical interest to a number of MITRE sponsors.  Pollen and spores are ubiquitous, naturally-occurring, microscopic structures that can act as taggants. At present, geo-location using forensic palynology (analysis of pollen and spores) is not common for a number of reasons including high cost, long turnaround time, and poor scalability.

Objectives
We will apply MITRE’s Computer Vision Toolbox (CVT) to develop automated methods to assist the forensics palynologist, particularly with minimizing the time-to-answer in pollen classification.

Activities
We have conducted a literature/vendor review on all modes of pollen ID and compiled available pollen distribution and morphological datasets applicable to automation. We obtained labeled pollen images from three palynology labs: Florida Institute of Technology, Texas A&M, and the University of Illinois (UI). We tested morphologically similar modern pollen grains at the genus levels using MITRE’s CVT, resulting in a best error rate of 2.03 percent. We will benchmark CVT relative to UI’s approach (e.g., Automated Recognition Layered Optimization) using images of fossil grains at both genus and species levels. We are developing a probabilistic Bayesian approach for geo-location based on normalized pollen taxa co-occurrences, as functions of geographic coordinates and elevation for modern surface pollen grains of the Neotropics. 

Impact
Computer-aided human-in-the-loop pollen ID and rapid geo-location analytics will enhance the government’s capabilities for tracing evidence in, for example, treaty compliance and origin information for seized drugs.

Public Release No: 13-0360

Problem
The short wave infrared (SWIR) portion of the electromagnetic (EM) spectrum, from ~900-2500nm, is an attractive spectral region to target for exploitation due in part to the abundance of spectral features exhibited by many materials of interest. The problem is that operating in the SWIR portion of the spectrum is very expensive in comparison to the visible region where use of consumer silicon sensors is possible. There is also the need to operate using an active source/detect methodology without emitting visible light. Currently, active spectral SWIR systems that do not operate in the visible spectral region use expensive focal plane arrays and heavy, inefficient bulb-based illumination with poor SWaP. Our research aims to solve these issues through the use of new types of commercial-off-the-shelf (COTS) LED and control electronics.

Objectives
We will develop a new active sensing architecture based on LED technology that can use spatial, temporal, and spectral techniques. This system will be able to operate in any portion of the EM spectrum that LEDs can; however, the SWIR spectral region will be targeted first. Our initial goals are to characterize current LEDs and perform a general state-of-the-art survey. Once we know the spatial, spectral, and temporal parameters, we will settle on a system design and create a proof of principle device. This initial device will open the door to many different design ideas, including a rugged ball device, a covert panel device, or even automotive lighting that smartly adapts to its environment.

Activities
The hypothesis is that a multispectral SWIR emitter and sensor can be assembled using COTS LEDs. First we will use a monochrometer to evaluate LED spectral characterization and perform sensitivity measurements. We will tabulate the results for individual LED types, and multiple LEDs will be combined, making a multispectral source/detector. A plot of wavelength versus radiance will show what gaps and peaks exist in the desired SWIR spectrum. A chopper wheel will allow us to determine the temporal response of the LED, which will allow us to determine a baseline switching time between the emit and detect modes. Following initial taxonomy, a 3D printer will be used for rapid design of a prototype system. This device should demonstrate the capabilities of the multispectral emit/detect LED device.

Impact
We will develop a new capability for multispectral illumination and sensing, providing control over several sensing aspects, such as spectral and temporal modulation, which will impact many types of current detection scenarios. Since LEDs inherently have a much lower SWaP than traditional hardware approaches to accomplish similar functionality, many new use cases can be envisioned.  As the project progresses, new applications will be identified that can benefit from the architecture of this system.

Public Release No: 13-0075

Problem
Video surveillence remains a critical capability for situational awareness. Video provides real-time, dense color and textural information over a wide area. Video camera technology is advancing rapidly, increasing frame rates, field of view, and pixel density. However, these advances are outpacing the ability to process the video information in a timely fashion, which reduces its usefulness. 

Objectives
To address this problem, we plan on using two techniques in machine vision to focus the needed processing on important regions in the image. The first is a technique used by the human eye, called "foveation," which is a variable resolution technique in which pixel density is greatest at the center and falls off rapidly. The second is learning "saliency" to understand where the propensity of activity typically occurs. Saliency drives the regions of attention, while foveation provides attentional focus. 

Activities
We are developing a guided foveation mask, which extracts information from key points in an image, and salience maps. The salience maps will be learned and adapted over time after initially being seeded by operator input.

Impact
We have completed the initial development of both a radial and cylindrical foveation filter and are working on the saliency region detector. This research could improve the ability to process video information.  

Public Release No: 13-0428

Problem
Analysts wish to characterize unknown authors according to latent characteristics, such as gender, age, and geographic affilation.  Recent success at modeling such author characteristics based only on informal text is typically limited by the requirement for strong similarities between training and test material.  As with many statistical machine learning problems, classifiers may perform quite well within a particular domain or genre, such as Twitter, but degrade sharply when applied elsewhere, such as blogs.  Similarly, trained models may degrade over time even within the same domain as key indicators and vocabularies change; this is especially problematic for social media sources.

Objectives
We are building on our strong successes with unknown author characterization to develop models that can be retargeted from one genre or domain to another. We will provide sponsors with the ability to train models on sources rich with supervised “gold standard” demographic information, and re-purpose them on novel impoverished sources without the usual degradation in performance. We will also develop mechanisms for making models more durable, so that their effectiveness decays less over time.

Activities
We are building datasets from a variety of social media sources to facilitate cross-genre and cross-domain experimentation. We are exploring a variety of approaches to statistical domain adaptation, beginning with the simplest methods, which essentially amount to editing or re-weighting an existing model’s features, to correct for simple distributional differences between the source and target domains.  We will also examine more sophisticated approaches, like the EASYADAPT or TrAdaBoost algorithms.  These constitute true domain adaptation and will allow us to learn new models based on little or no labeled target domain data.

Impact
Analysts will be able to reuse models on new sources and genres with a minimum loss of accuracy, allowing for more rapid ramp-up on new sources.  Models will decay less over time within a domain, requiring substantially less maintenance and retraining.  We have spoken to a number of sponsors who have expressed strong interest in automatic characterization of unknown author demographics. This work will make these capabilities much more dependable and operationally useful.

Public Release No: 13-0594

Problem
Policy considerations, such as carbon taxes and aircraft CO2 standards, are intended to reduce aviation climate impacts. For the airlines, these policies introduce new tradeoffs between fuel efficiency and maximum throughput, which affect each company’s bottom line.  Given that aviation environmental regulations could change the NextGen value proposition for airlines, the aviation community needs to examine the NextGen business case implications of climate policy.

Objectives
In our research we will formulate environmental regulation as policy scenarios that introduce business or operational contraints, and we will model a range of airline responses using data on observed behaviors.  We will translate the airline responses under each policy scenario into a new set of National Airspace System (NAS) operational and environmental metrics, leveraging MITRE’s aviation simulation analysis capabilities.  The updated metrics will reflect changes in NextGen stakeholder valuation and business cases.

Activities
We will begin by defining aviation climate goals, major progress drivers, and a set of environmental regulation policy scenarios. We will develop an operator response model and operational and environmental models, which will provide estimates of the impacts. The resulting NAS performance metrics will be interpreted in the context of the NextGen business case.

Impact
This research effort will help inform NextGen decision making  and implementation strategies through better understanding of the changes in NAS performance and NextGen stakeholder valuation that could occur in the presence of more environmental regulation. 

Public Release No: 13-0728

Problem
Technology developers and emergency response managers need methods to assess the effectiveness of operations in simulations and exercises. In FY12, we conducted the MITRE Citizen Emergency Response Portal System Simulation Experiment (CERPS SIMEX) to examine the use of social media to obtain citizen input during emergency response operations. This team will analyze communications produced by the emergency response managers who participated in the CERPS SIMEX to develop measures that will enhance understanding and evaluation of the use of CERPS technology in emergency management operations.

Objectives
The team's goal is that participants in emergency management simulation experiments and exercises will have informative measures based on the structure and content of communications in the simulations. Objective measures based on communication patterns and language content can provide (1) insights into operational processes that would be difficult to observe without systematic analyses and (2) objective evidence to support conclusions based on anecdotal observations.

Activities
A variety of measures based on log records of the communications in the experiment have been computed. Social network analysis will model the number and types of messages sent among participants during response operations. Measures based on analysis of linguistic features will reflect properties of the communication content. All of the measures will be computed for sequential time intervals in the simulation and compared to the simulated events occurring during those intervals. A visualization application will be developed to provide dynamic representations of the networks of communication relations.

Impact
The metrics we develop will allow emergency responders to glean more useful information from expensive simulations and exercises, and technology developers will be able to quantify the impact of using new tools and applications. These advantages are not limited to the emergency response community; it will be easy to adapt the metrics to other real-time response environments, such as operations conducted by the military or by intelligence agencies.

Public Release No: 13-0371

Problem
Complex military systems demand increasing levels of performance, technology integration, and power savings for applications constrained by cost, schedule, and the protection of critical information. Programs of Record in areas such as radar, navigation, and tactical radios are starting to expect the same data fusion capabilities found in low power, ubiquitous commercial personal devices (e.g., smartphones and tablets), a trend that requires an increasing amount of memory, processing resources, and input/output (I/O) bandwidth. The end users demand solutions that provide similar data fusion capabilities with size, weight, power, and cost (SWaP-C) comparable to commercial applications.  Many programs choose commercial-off-the-shelf components, such as FPGAs, at the expense of increased SWaP-C.  The use of Application-Specific Integrated Circuits (ASICs) implemented in advanced semiconductor process technologies offers opportunities to improve energy efficiency, security, and performance. For many years, MITRE has been applying advances in semiconductor process technology to deliver leading-edge solutions for our sponsors. A current survey of the intellectual property (IP) offered at a Trusted Foundry (TF) reveals that, while the semiconductor technology is capable of high-speed processing for high-bandwidth applications, the interfaces necessary for data flow on and off chip are not readily available.

Objectives
Our solution is to develop high-bandwidth IP and techniques that are portable across semiconductor processes, cost effective, and applicable to multiple programs. The capability to integrate state-of-the-art IP for high-speed I/O and high-density memory with Silicon-On-Insulator (SOI) semiconductor process technology will allow MITRE to better assist sponsors in the development of program acquisition strategies. Our sponsors leverage these advanced risk reduction prototyping facilities to validate technical requirements and conduct lab and field testing during the technology development phases of their programs. Adding the high-bandwidth capability to our microelectronics development environment will also enable MITRE to assist a broader set of government sponsors. As part of this research, we will collaborate with the TF program, communicating the needs of our sponsors to assist them in their selection of technology, IP, and development of roadmaps.

Activities
In FY12, this project team developed two high-speed interface techniques that are independent of semiconductor process and can be ported across multiple technologies. The techniques were coded and verified, reaching input bandwidths of up to 1GHz when interfaced to a typical Analog-to-Digital Converter  simulation model. This year, we will implement these ideas and integrate a process-independent DDR2 memory interface in 45nm SOI technology. This enhancement will provide three new interfaces for a reduced SWaP-C radar application.

Impact
We will identify semiconductor process technology and techniques and transition them to government programs to ensure that the warfighter will have a technological and tactical advantage. This research will also help government programs reduce costs, schedule, and risk. In past years,  this team has provided sponsors with advances in IP, design and verification methods, including a radar pulse compression engine ASIC for a system, which prompted the sponsor to fund a reduced SWaP-C version. MITRE has also identified several other applications of this technology and is working with sponsors and programs to provide new innovative solutions.

Public Release No: 13-0653

Problem
A key challenge to effectively treating chronic disease is early detection, combined with appropriate treatment to reverse, arrest, or retard the disease process. Current diagnostics are expensive, invasive, specialized, and often inconclusive until the late stages of the disease. As a result, screening techniques are not routinely used, disease detection is significantly delayed, and treatment options are limited.

Objectives
There is growing evidence indicating that subtle retinal changes are correlated with the onset and progression of several chronic diseases. There have also been significant advancements in retinal imaging technology with respect to higher image resolution, larger field of view, 3D visualization of internal retinal structures, and greater ease of use. This project team is testing the hypothesis that automated diagnostics using retinal imaging could provide a non-invasive, safe, and scalable means for early detection of a broad class of diseases.

Activities
In the past two years, this team developed the initial open source platform for retinal imaging research, including a database of retinal images and image processing algorithms for diabetic retinopathy detection, and developed image processing algorithms and a clinical decision-support aid. In FY 13, we will: complete development of the algorithm and clinical decision-support aid; conduct a diabetic retinopathy screening study to validate algorithms; and transition the technology to sponsors and industry.

Impact
Initial studies show that automated diagnostics is a promising approach for diabetic retinopathy screening. Using this approach for diabetes, as well as other chronic diseases, may allow identification of subtle changes that serve as early indicators of health issues.

Public Release No: 13-0635

Problem
The financial system is complex and adaptive, with many underlying components and non-trivial feedback loops. Modeling and simulation is one way to understand and quantify complex phenomena. There are many models in academia and industry that investigate the dynamics of the financial system; however, each model carries a unique set of assumptions, abstractions, and behaviors. To effectively and quantitatively understand sources of risk in the financial system, financial regulators will benefit from a capability that takes a holistic look at multiple models across metrics and statistics that describe system behavior. The aim of this project is to provide a flexible and robust capability to host these financial models, execute them in a high-performance computing environment, and analyze, compare, and contrast them using unique methods.

Objectives
We will develop infrastructure and analytic methods that facilitate quantitative, data-driven approaches to modeling, analyzing, and understanding behavior in the financial system. In FY12, this team implemented a prototype environment that hosts, executes, and analyzes financial simulations using the MITRE Elastic Goal-Directed (MEG) Simulation Framework. MEG is a middleware capability that provides a number of powerful features for running simulation experiments, including job scheduling services on high-performance clusters, design of experiments, and data-storage capabilities. Once the models are hosted and executed, we will implement a robust analytical capability to validate and understand the time-series outputs from the various simulations.

Activities
We plan to use our prototype environment to build on the work done by the team in behavioral finance, market models, and validation. Our goal is to extend and solidify our validation for use with general market models. This investigation includes researching, implementing, and testing the application of the statistical tools across a wide range of market models and conditions.

Impact
The Financial Modeling and Simulation Execution Environment is a decision-support tool and process to test and validate models and assess the plausibility of model results. This tool also has the ability to calibrate model parameters and an interface to facilitate analysis and presentation of model dynamics and output. This capability will provide regulators with a scalable tool to understand and experiment with various models at their disposal.

Public Release No: 13-0639

Problem
No mid-term automation capability exists for storing and processing Unmanned Aircraft System (UAS) contingency routes. Contingency routes are plans that would be executed in case of a loss of the command and control (C2) link between the pilot’s control station and the aircraft, or some other unexpected event. As a result, controllers have limited situational awareness and cannot depict an unmanned aircraft’s flight route on air traffic management displays during a contingency event.

Objectives
This project aims to re-purpose a mid-term capability to facilitate coordination of UAS contingency routes between remote pilots and the air traffic management (ATM) system. The outcome of this project will be an operational concept and potentially a demonstration of how the Trajectory Option Set (TOS) message could be leveraged by the remote pilot to communicate contingency routes. We will investigate augmentations to the existing TOS specification or changes in TOS behavior/procedures to accommodate UASs.

Activities
We will investigate repurposing existing messages to demonstrate a contingency planning capability for UASs. The team is currently developing an initial operational concept that will describe how, prior to launch, a UAS could file its primary and contingency routes using the TOS message. In a contingency event, the UAS could send an updated TOS message containing only its active contingency route. The flow manager could then take appropriate action.

Impact
Leveraging a mid-term capability such as TOS (with essentially no modifications other than UAS-specific procedures) to handle UAS contingency routes would help solve a critical integration issue for UAS (lost-link), while providing greater predictability, consistency, and situation awareness for controllers while enabling the use of ATM decision-support systems.

Public Release No: 13-0771

Problem
Generating shared keys on-the-fly is seemingly impossible in the following sense: Alice and Bob are two agents who wish to send each other messages, the content of which should remain secret to all other observers. For Alice and Bob to agree on a key without revealing it to eavesdroppers, it seems that the agents would first need a secure channel for negotiating the shared key. To bypass this chicken-and-egg situation, the current paradigm is to generate keys in advance and manage them as they evolve (i.e., are created, deleted, or updated).  Such systems can be problematic, however, because they are rigid, requiring pre-placed keys and/or because key management is cumbersome. To illustrate this problem, consider the U.S. Army’s radio network.  First, the network is huge; the number of radios is many times greater than the number of personnel.  Second, the radios require Type-1 keys, which are regularly regenerated and manually redistributed. This makes key management prohibitively expensive.  To make matters worse, physical keys are easily lost or misplaced.  In such circumstances, sensitive and mission-critical communications are sometimes not sent or are sent in the clear.

Objectives
We believe that on-the-fly shared key generation is possible.  Even without any a priori agreement, Alice and Bob can access a common source of randomness: the physical environment in which they are operating. They can each extract the same shared key from their respective measurements of this source.  The secret key is derived from the shared key by using a chaotic map, which is analogous to dropping the insecure bits of the shared key.  Our objective is to enable on-the-fly, secret key generation that can then be used as a security primitive.  Our goal for FY13 is to demonstrate the practicality of physical-layer security via radio experiments, which will determine the achievable “multipath signal” and “reciprocity” levels of communication channels.  If the experimental results validate physical-layer security, the next step will be to design a system that relies on the mathematical properties of a chaotic map to generate statistically secure keys. 

Activities
Our planned activities include: -Experiment 0, for base-lining (e.g., determining the transmit- and receive-transfer functions); -Experiment 1, for determining the multipath signal level and coherent integration time; -Experiment 2 for determining whether channels are truly reciprocal over short time durations; -Socializing our approach, including submission to IEEE Globecom13.  

Impact
If we are successful and can demonstrate that RF communication channels support shared key generation, it will be possible to generate keys on-the-fly as needed, which may eliminate the need for key management altogether in certain applications.  Moreover, less a priori setup will be needed, which will give users more flexibility. 

Public Release No: 13-0139

Problem
The intelligence community and military have recently acknowledged that climate-induced prolonged drought, water scarcity, and national security are highly linked.  If the climate situation deteriorates, resulting in water and food scarcity, and geopolitical instability, then multiple state, nonstate actors, and even business entrepreneurs may implement a rogue clandestine geoengineering effort in desperation, or possibly as an act of aggression. GeoEngineering (GeoE) refers to those technologies that mitigate the climate change effects of excessive greenhouse gases (CO2, SOx, NOx, CH4); solar radiation management; carbon sequestration (CO2 capture); reduction of ocean acidification; and weather modification. However, climate change information is massive, complex, and widely dispersed, and is not organized to support decision making, policy, or intelligence analysis.

Objectives
To address urgent and complex national and transboundary situations that may suddenly arise from global warming, this project will create a GeoE knowledge base and an evaluation to support analysis, policy formulation, and decision making by U.S. policy makers, analysts, the military, and the State Department. The knowledge base will contain key ideas and sources. The multifactorial evaluation framework will describe GeoE technologies, readiness, decisions, costs, and consequences (transboundary, political, military, social, and economic).

Activities
We are developing the framework and knowledge base in conjunction with government subject matter experts and with Dr. Kerry Emanuel, MIT Department of Earth, Atmospheric and Planetary Sciences. We will also hold a small workshop with  a range of experts to brief government staff, gather more feedback, and discuss future actions.

Impact
We will document the results of our work and the workshop and share these reports with sponsors and other appropriate stakeholders.

Public Release No: 13-0692

Problem
Stabilizing the security sectors of countries in democratic transition, such as those involved in the Arab Spring, is very important to our national strategic interests. The prevailing lack of security governance that sometimes plagues these aspiring democracies can result in an increase in violent extremism and the potential insecurity of weapons of mass destruction. Many of these security establishments must be radically transformed to support stable democracies. The U.S. government can play a critical role in building our allies' capacity, especially through partner state security sector reform and development programs. However, most U.S. security-related interagency frameworks and policy guidelines are limited in scope and scale, and typically require  major resource investments to initiate security sector reform programs. The government needs a low-risk capability that can accelerate and inform security problem solving and decision making during times of heighted uncertainty.

Objectives
Our goal is to develop a dashboard-based analytic capability that offers U.S. government analysts early detection of security signals and their rapid response to security vacuums. The customized dashboard can be used to define, develop, inform and influence security topics of critical national importance.

Activities
We will develop an underlying capability and associated dashboard that measures sentiment, attitudes, and perceptions of key events in one region. It will include psycho-social, political, and economic data about security sector governance and performance, such as corruption, transitional justice, police reform, democratic civilian reform, civil society, and economic development. We will then present the “social radar” dashboard capabilities to U.S. government agencies, as well as policy, research and academic institutions.

Impact
This tool will apply MITRE’s social radar capabilities to a critical mission space and inform government decision making during uncertain situations.

Public Release No: 13-0536

Problem
Civilian agencies lack a well-defined systems engineering process for evaluating changes to public-facing services. Finding ways to transform service provision that will increase the quality of the service or decrease the costs is critically important.

Objectives
Our research seeks to apply proven Service Science Management and Engineering (SSME) methods from industry to the analysis, evaluation, and improvement of public-facing federal services. We will analyze the public-facing services provided by several agencies using SSME methods, develop alternative methods of provisioning these services, and provide realistic cost/benefit analysis of the risks and opportunities provided by changing aspects of public-facing services.

Activities
In FY13, we will focus on public-facing services provided by the Internal Revenue Service, Department of Veterans Affairs, Centers for Medicare & Medicaid Services, and U.S. Census Bureau.  The services involving direct interactions with the public will be cataloged and analyzed using SSME methods to identify potential changes that could significantly improve quality, reduce cost, and eliminate rework. Our initial survey of 24 top federal agencies revealed six common public-facing business processes.  Likenesses among these transactions across multiple agencies lends credence to the hypothesis that there are fundamental similarities in the business processes performed by many federal agencies.

Impact
We have developed a general model of the federal government service system ecosystem and have validated it with use cases for public-facing transactions from two agencies.  The project will give government agencies a repeatable methodology for critically examining public-facing services and a systematic approach to improving service quality and reducing the cost.

Public Release No: 13-0227

Problem
The IRS's most recent estimate of the tax gap is $376 billion. Reducing this gap is difficult, as the types of schemes to avoid taxes are subtle, distributed, and change over time. To address the complexity of this problem, the IRS has started using graph structures to model the data. Querying and visualizing this graph structure has provided great insight into the complex web of tax entities that fraud analysts must search.  However, these tools only marginally support insight into how tax structures evolve over time. The goal of this research is to build an analytic tool that will enable the detection of interesting graph changes.

Objectives
Our goal is to provide analysts with a tool that allows them to quickly see  how tax entities have changed over some period of time.  By summarizing these changes using a vocabulary that is meaningful to analysts, we think important changes in tax structures will be quickly identified and those with tax compliance issues will be quickly found.

Activities
This year we are developing tools for identifying and  summarizing temporal changes in graph data. We are also gathering feedback from IRS analysts on appropriate terminology and visualization for temporal changes.

Impact
This tool could help IRS analysts discover previously undetected tax-evasion behavior

Public Release No: 13-0621

Problem
Current health information exchange standards are unnecessarily complex, inflicting high software development costs and imposing significant barriers to entry. If health data is not exchanged in a timely manner, medical errors can occur, costs can escalate, and the continuity of care can be interrupted. Moreover, patients can feel estranged from their own medical care because of the difficulty of obtaining their own medical records in an easy-to-understand, integrated, secure form. If an improved generation of health exchange standards supported bidirectional mobile information access, then major healthcare cost contributors such as chronic disease management and aging in place could also be addressed.

Objectives
Our objectives for hData are to: -Make adoption of health IT standards easier by using RESTful web services as a single end-to-end standard, replacing the cumbersome SOAP and WS-* stack. -Be mobile-device friendly by focusing on web technologies (HTTP, XML, and JSON) and disaggregating clinical information to allow exchange of information in small pieces, rather than requiring an entire patient record to be copied from system to system. -Employ a resource-oriented framework, where each piece of clinical information is identified by URL, and exchanges are discovered and coordinated by means of published resource profiles. -Separate the content model from exchange mechanics, allowing clinicians and content holders to design resources and wire formats separately from transport methods.  -Approach security via widely-used web authentication and authorization standards, such as Open ID and OAuth 2.0.

Activities
We have designed a set of standards for lightweight, scalable clinical information exchange. Two specifications have been developed, the hData Record Format (HRF), which was approved as a Draft Standard for Trial Use by HL7, and the hData RESTful Transport (HRT), which was recently approved as a Normative Specification by OMG. Efforts are under way to develop the normative specifications for HL7. These standards specify network-facing interfaces based on REST and Atom that can be layered on existing clinical systems, or added to mobile devices, to achieve homogeneous, end-to-end information flow. The hData standard organizes clinical information for web access, provides a pub/sub model for data exchange, defines web methods for consuming and producing data, standardizes metadata annotation of data, and specifies self-description of hData-enabled sites. We are investigating bidirectional flows using messaging-oriented middleware.

Impact
hData has been adopted inside and outside of MITRE. Transition activities with ONC resulted in a project with the Federal Health Architecture program, in which we conducted pilots for Army Telemedicine and the Maine Health Information Exchange. RESTful Health Exchange (RHEx) demonstrated how popular authentication and authorization models, such as OpenID and OAuth 2.0, can be used with hData. The hData team is focusing on several new initiatives: developing device interoperability standards with the Continua Alliance, aligning with an emerging electronic medical records standard at HL7 (FHIR), and achieving normative status at HL7. Within MITRE, hData has been used by projects such as hReader and medCafe. By significantly shifting the technology landscape, hData is helping sponsors and the private sector evolve toward patient-centric, cost-effective healthcare.

Public Release No: 13-0131

Problem
Today, patients do not have access to their comprehensive health history at Health Information Exchanges (HIEs). We will demonstrate how patients can be empowered by gaining access to their comprehensive health history in the Maine HIE by using mobile devices.

Objectives
Our goal is to demonstrate how patients in Maine can be empowered by gaining access to their integrated health history at the Maine HIE Clinical Data Repository using mobile devices. We believe that by securely accessing their health history, patients will actively engage and participate directly in their own care. At the HIMSS Interoperability Showcase, we highlighted integration of the following capabilities: -RESTful Health Exchange (RHEx) technology, developed by the FHA and MITRE, piloted with HealthInfoNet and Islands Community Medical Center -Integrated patient history at the Maine HIE, developed by HealthInfoNet, with participating providers, Franklin Memorial Hospital and Islands Community Medical Center -hReader mobile platform, developed by MITRE.

Activities
Key activities on the project include: -Develop realistic scenario to demonstrate the capability -Build custom RHEx client to allow secure data sharing between the Maine HIE and hReader -Test and integrate the capability -Demonstrate patient access to their integrated health history at the HIMSS Interoperability Showcase.

Impact
Our goal is to give broader exposure to RHEx and hReader technology at the conference and identify additional applications for the capabilities with other stakeholders.

Public Release No: 13-0994

Problem
Healthcare sector leaders want "meaningful use," a goal that requires a more open flow of shared healthcare information. However, daily data breaches and other security and privacy incidents across organizations undermine confidence that information is actually being protected. Due to issues such as the need for granular control and the increasingly complex topologies of automated systems, many see the efforts to achieve meaningful use as being severely hampered by the lack of credible information protection.

Objectives
This project team aims to design a complementary approach for information protection. We will leverage existing work by MITRE (including hData, RHEx) and other technologies and add the relatively new, but vetted, technology of attribute base encryption. This holds the promise of offering the information-level security and privacy protection demanded, while also supporting granular control. We will collaborate with external organizations to develop designs and engage with sponsors on  impact and transition.

Activities
We will create technical designs that integrate attribute based encryption  into existing  RESTful secure technologies, such as RHEx.  We will also enlist collaborators, including those participants in the Strategic Healthcare IT Advanced Research Projects on Security (SHARPS) project  to help refine those approaches.  In addition, we will work with one or two identified sponsor initiatives and influence outcomes for improved implementation of meaningful use.

Impact
The successful transition of this work to sponsors should lead to improved guidelines, standards, and recommendations that better address information protection and granular control, and thus better fulfillment of meaningful use. The technology transition should also lead to suppliers offering better solutions to the healthcare sector.

Public Release No: 13-0732

Problem
Roughly four out of five healthcare dollars are spent on people with chronic conditions.  Even more than the daily management of their illnesses, chronically ill patients and their caregivers struggle to evaluate and communicate their symptoms, comply with treatment plans, and coordinate among multiple providers.  Too often, providers must make critical decisions based on limited, incomplete patient-provided data. We believe each patient needs to contribute to his or her care team.

Objectives
The healthAction Patient Toolkit leverages mobile IT so that chronically ill patients can track longitudinal data (e.g., symptoms), communicate effectively with their providers, improve their situational awareness, and increase treatment compliance. Our research will identify ways to maximize patient engagement, assess the value of patient-supplied data, and develop habits that improve health outcomes without adding a burden to the patient. 

Activities
We are completing development of the healthAction Patient Toolkit iPad application, incorporating other MITRE research projects to ensure security and connectivity. Working with the University of Virginia, we will objectify and quantify the reliability of patient-generated symptom data and measure the value of this data to providers. A longitudinal trial will evaluate the impact of this tool on the personal and financial burdens of chronically ill patients. We will transition the system and our lessons learned via open-source and publication as appropriate. 

Impact
An initial prototype of the healthAction Patient Toolkit is being readied for evaluation.  This tool increases the ability to track changes in a patient’s health, ensures patients effectively communicate with their providers, increases treatment compliance rates, and enables providers to make decisions based on complete sets of information. Chronically ill patients can then partner with their providers to improve health outcomes and reduce costs.

Public Release No: 13-0629

Problem
Many algorithms and systems are "black boxes," meaning one can know the inputs they use and the outputs that are produced but one cannot directly examine the inner workings.  Unfortunately, many of these black boxes must be well understood as they may have significant impact on our nation. Examples of potentially important black boxes include such things as high-frequency trading algorithms, UAS guidance systems, and sensor data fusion systems.  In order to adequately test these sorts of black box systems, we need a method designed to manipulate the system's inputs and outputs while remaining agnostic to the "interior" of the system.  The basis of such a method was introduced by Dr. John Miller, known as Active Non-linear Tests (ANTs). 

Objectives
Heuristic Black Box Testing (HBBT) is implementing and extending the Miller ANTs methodology to work on modern computing infrastructures and utilize additional heuristic search algorithms.

Activities
HBBT has begun to implement the ANTs algorithm and integrate it with the MITRE Elastic Goal-directed Simulation Framework (MEG).  The MEG is a web-based system for designing simulation-based experiments, running these experiments in a high performance computing environment, and gathering the results.  A key component of the MEG is the ability to perform designs of experiments that utilize biologically inspired search algorithms such as ANTs.  Furthermore, we have defined an initial docking exercise to check our implementation against Dr. Miller's published results. Moving forward in FY13 HBBT will finish the initial implementation of the ANTs algorithm and dock it against Miller’s original work.  Once satisfactory docking is achieved HBBT will extend the algorithm to include more heuristic search options.  We are in the process of identifying a set of test cases on which to use the HBBT methodology.  Some test cases under consideration include UAV control algorithms and high-frequency securities trading algorithms. HBBT has begun to implement the ANTs algorithm and integrate it with the MITRE Elastic Goal-directed Simulation Framework (MEG).  The MEG is a web-based system for designing simulation-based experiments, running these experiments in a high performance computing environment, and gathering the results.  A key component of the MEG is the ability to perform designs of experiments that utilize biologically inspired search algorithms such as ANTs.  Furthermore, we have defined an initial docking exercise to check our implementation against Dr. Miller's published results. Moving forward in FY13 HBBT will finish the initial implementation of the ANTs algorithm and dock it against Miller’s original work.  Once satisfactory docking is achieved HBBT will extend the algorithm to include more heuristic search options.  We are in the process of identifying a set of test cases on which to use the HBBT methodology.  Some test cases under consideration include UAV control algorithms and high-frequency securities trading algorithms.

Impact
If successful, HBBT will create and demonstrate the viability of this methodology to be used to understand the behavior and potential failure modes of systems that cannot be examined directly.

Public Release No: 13-0776

Problem
The fragmentary and stove-piped nature of healthcare IT solutions today often results in incompatible infrastructures, non-standard user interfaces (UIs) and significant investment in software and services to enable discrete systems to communicate with one another. The result is Electronic Health Records (EHR) UIs that have a "lack of usability in non-standard EHR user interface design."

Objectives
The Open Health Tools (OHT) consortium is planning to address this problem by integrating open source solutions into an end-to-end capability that is adaptable to health organizations’ specific needs and resources. In addition, any OHT solution can be leveraged by the Open Source Electronic Health Record Agent community. MITRE's hOpen enables this adaptable health IT Infrastructure by working in collaboration with OHT to provide a composable UI as part of a suite of OHT and MITRE tools, which will result in an open source pilot of end-to-end capability.

Activities
hOpen will integrate medCafe with other relevant open and standards-based MITRE projects (such as hData and RHEx), in collaboration with OHT, to provide end-to-end capability. In FY 13, OHT will stand up a new UI platform that will integrate multiple systems. OHT plans to leverage medCafe as part of this innovative platform to provide an adaptable and intuitive UI capability. This will showcase alternative approaches to visualizing and adapting UIs to meet diverse needs across health organizations.

Impact
We have integrated hOpen with the RHEx restful data server, using hData with the Green Clinical Document Architecture (CDA) format for interoperability. hOpen also demonstrates how an open source version of the Department of Veterans Affairs health system can be integrated with hData, using the Green CDA format to display health information through a composable interface. The next steps will involve integrating the security features of RHEx to provide secure end-to-end capability among diverse health infrastructures, such as a Mongo database and a MUMPS-based health system. The system will then integrate with the newly developed Runtime Platform from OHT to display an end-to-end capability. Tools will be available and discoverable through the OHT-developed HingX web site.

Public Release No: 13-0661

Problem
Currently, patients don’t own and often can’t access their own health care data.  When the data is available, it is often poorly presented and hard to understand. Finally, most patient-provider communication is very seasonal as opposed to on-demand, as needed. hReader seeks to address all of these problems.

Objectives
hReader is a secure, patient-centric, open source mobile health data manager that provides families with all of their health information.  A built-in applet framework provides hyper-personalized capabilities to engage patients with their health status and empower them to improve their health.  hReader’s goal is to demonstrate that mobile technologies can empower patients to improve their health.  Storing all of their electronic health record data along with their personal wireless sensor data on a secure iOS platform allows families to access their information anywhere, even at a provider’s office without an Internet connection.  Complete health data records can be sent to providers, clinical decision systems, and popHealth for timely, personalized recommendations.

Activities
hReader is currently being evaluated by state health information exchanges. We are developing the two-way Blue Button communication protocol between providers and patients and also building an open-source development community so that outside developers can innovate on the hReader platform at hreader.org.

Impact
If successful, hReader will be a disruptive game-changer, shifting health data ownership from providers to a shared-ownership model between providers and families.  hReader presents a family’s health data in a compelling design that increases health awareness and should drive habits to improve health outcomes at lower total cost.

Public Release No: 13-0205

Problem
Many fields of science and engineering routinely benefit from the use of microscopy.  Hyperspectral remote sensing of the earth should be no exception—yet sample characterization at the microscopic scale is not routinely conducted.  The remote sensing community continues to recognize that detailed sample characterization is lacking in its current best practices. Microscopic characterization of materials of interest to our sponsors will complement current macro-scale (i.e., hand-sample sized) measurements.

Objectives
The overarching goal of the hyperspectral imagery (HSI) microscopy research is to fill a gap in "ground-truth" data collection known to exist within the remote sensing community.  Microscopic characterization complements current point spectrometer measurements and deepens our understanding of the origin, nature, and variability of spectral signatures of materials of interest to MITRE's sponsors.

Activities
We will: -Build a visible through shortwave infrared HSI microscope based on a tunable laser illumination source -Develop a VNIR/SWIR spectral measurement laboratory as a resource for the DoD/IC -Continue spectroscopic investigations on particular materials of interest relevant to the remote sensing programs of our sponsors -Develop a data simulation capability prototyped in FY12 -Continue modeling the radiance field/spectral mixing -Develop an uncertainty analysis (error propagation) for imaging spectrometry resulting in a protocol for incorporating pixel level uncertainties into the analysis and exploitation of HSI data.

Impact
We will enable the production of high quality data from a spectral measurement capability that has not been used before. The tunable laser-based spectrometer will provide a high signal-to-noise ratio VNIR/SWIR measurement capability, high spectral resolution (when needed), combined with high spatial resolution imagery.  The remote sensing community will also have a better understanding of the origin and nature of spectral signatures, and community spectral signature databases will have more robust metadata due to a more detailed characterization of the sample(s) measured.

Public Release No: 13-0181

Problem
Current approaches to network defense focus on identifying and mitigating vulnerabilities; they are reactive and susceptible to previously unseen threats. What if you could have an enterprise network that protects your user workstations, makes your infrastructure inaccessible to attack, eliminates anonymous traffic, enforces your “permissible use” policies, is predominantly self-configuring, significantly increases situational awareness, and preserves your investment in existing infrastructure? MITRE designed the Identity-Based Internet Protocol (IBIP) network with these goals in mind.

Objectives
Our IBIP approach assumes that adversaries will penetrate our networks and proactively reframes the problem space to one in which we, not the adversary, have control. IBIP accomplishes this by establishing and enforcing permissible-use policies that  specify what the hosts and users on the network are permitted to do in support of the mission, rather than chasing undefined anomalous activities. The policy monitoring and enforcement are carried out by the infrastructure via the deployment of a policy enforcement point (PEP). This PEP is responsible for hiding all clients, making the infrastructure inaccessible, ensuring that all servers behave within their approved operational limits, executing access control policies, and reporting all policy violations (PV). An accumulation of PVs can affect “trust” for that user or host, which can be used to further control access. The PEP also ensures that all network traffic contains identity information for the host and user (and, optionally, organizational affiliation as well as current role and level of trust).   The immediate benefit of IBIP is a dramatic reduction in an adversary’s access to vulnerabilities; adversaries are forced to deduce and conform to patterns of permissible behavior on a host, user, or organizational basis. IBIP makes reconnaissance actions by adversaries much more challenging and visible by monitoring and reporting violations of the permissible use policies. IBIP is agile and responsive to evolving threat conditions, with pre-established responses to policy violations that are aligned with the current threat condition. IBIP currently supports up to five Information Operations Condition states permitting, for example, permissive policies in some situations and more restrictive ones in other situations.

Activities
We have been developing IBIP since October 2010; currently, we are running an operational pilot within MITRE.  Phase 1 of the pilot supports five IBIP “cells” (in Bedford, MA, and McLean, VA, locations), which collectively constitute a single IBIP enclave. A cell is an incrementally deployable local area network segment that could support more than 200 users. Phase 2 of the pilot will involve a potential ten-fold increase in users and the addition of one or more remote sites.

Impact
IBIP's capabilities--which are applicable to any organization with assets that require protection--establish it as a potential game-changer in cyber defense.

Public Release No: 12-5161

Problem
Foreign language data is often paper-based and optical character recognition (OCR) of the pages needs to be performed to convert images of the documents to electronic texts so that  they can be processed by the tools that allow analysts and translators to handle increasing amounts of information.  Even though OCR has made great strides for languages, such as Arabic or Urdu, poor quality document images are common and cause accuracy to plunge. For effective downstream analysis of linguistically diverse data, the output of OCR must be improved.

Objectives
We will use machine translation (MT) software to translate from poor quality OCR output to better quality OCR output.  MT software is designed to translate between human languages such as Arabic and English.  Training an MT system to translate from the output of OCR to higher quality language output builds on a proven statistical technique to address this problem in a novel way.

Activities
We selected two corpora to train and test the system:  Army field manuals and United Nations proceedings. We then used software to systematically degrade the document image quality.  After these are OCRed, the inaccurate outputs of the OCR will be aligned with accurate "gold standard" versions.  Then the statistical package can be trained to convert errors into correct text.  We will test the resulting system against gold standard texts, and the impact on downtream processing (MT) will be tested as well.  After proving the technique in Arabic, we will select new languages to test.

Impact
The completed corpora will be released to the community for OCR testing and development. These techniques to improve the automated processing of image data should result in a new system that has the potential to improve the downstream analysis of millions of documents.

Public Release No: 13-0599

Problem
Systemic risk analysis poses daunting information management challenges to financial regulators. Heterogeneous data must be transformed and integrated to meet the needs of diverse risk analysis models, many of which are computer-based simulations. These data management tasks are often performed with custom code. With hundreds of analysts creating tens to hundreds of thousands of simulation runs on thousands of overlapping data sets, it becomes extremely difficult for analysts at regulatory agencies to find relevant data and models, understand how different simulation results were obtained, and find likely collaborators based on who else is using similar data and models.

Objectives
To address this challenge, MITRE is building a prototype data management infrastructure for financial systemic risk analysis. The toolset includes a metadata repository for data and simulation models, software for automatically capturing data provenance (i.e., data’s derivation history) from simulation executions, and an open source data transformation tool.

Activities
We are: 1) completing provenance capture from an open source data transformation tool (Pentaho Kettle), 2) developing an analyst interface to support registration and search of data and models as well as query of provenance, 3) integrating these capabilities with the MITRE Elastic Goal-directed simulation middleware, and 4) publishing lessons learned and sharing with sponsors.

Impact
Our tools should greatly improve financial analysts’ ability to find and understand the data used by different simulation runs. The research has generated significant interest among researchers and sponsors. We have published two chapters in the Cambridge University Press Handbook on Systemic Risk and presented papers at the International Conference on Information Quality and the International Working Conference on Enterprise Interoperability. We have presented our work to the Commodities Futures Trading Commission and the Intelligence Advanced Research Projects Activity. Additional discussions with the Securities and Exchange Commission, Office of Financial Research, and other regulators indicate broad interest in this work.

Public Release No: 13-0333

Problem
In aviation human-in-the-loop simulation studies, in which a controller issues clearances to simulated aircraft, a human simulation pilot (“simpilot”) acts as pilot for the aircraft by reading back clearances and commanding the aircraft. Low or inconsistent simpilot proficiency, particularly under the high traffic levels used in NextGen studies, can cause controllers to change their behavior, negatively impacting experiment results and leading to faulty conclusions. Consistent and proficient simpilot performance is needed to ensure valid experimental results.

Objectives
Our research aims to construct an interactive simpilot agent using  automated speech recognition (ASR) and generation. Recognizing that these technologies fail at times, the concept calls for a human simpilot to monitor the agent and gracefully intervene when a failure occurs. For studies requiring multiple controllers, a single human simpilot may be able to monitor multiple simpilot agents.

Activities
CAASD’s standalone ASR-based en route phraseology trainer has been migrated to a general lab capability and work with the lab audio infrastructure. Having identified that ASR techniques used for phraseology training are inadequate for the operational context, we developed a modified grammar and new statistical language model that show promising preliminary recognition rates. We also modified the standard simpilot aircraft control interface to accommodate the task of a human monitor.

Impact
The end result will be an agent whose performance is consistently proficient and controllable. The impact will be a reduction in simpiloting costs, greater flexibility in performing studies, and greater trust that NextGen studies are free of a potential experimental confound created by the human simpilot. If our work is successful, there will be technology transition opportunities to national and international aviation research and training institutions and facilities.

Public Release No: 13-0182

Problem
Today, while iOS meets the enterprise security needs of customers, many security experts cite critical vulnerabilities and have demonstrated exploits, which pushes enterprises to augment iOS deployments with commercial solutions.  MITRE created the iMAS collaborative research project to protect iOS applications and data beyond the Apple-provided security model and reduce the adversary’s ability to perform recon, exploitation, control, and execution on iOS mobile applications.  iMAS will transform the effectiveness of the existing iOS security model across major vulnerability areas, including the System Passcode, jailbreak, debugger/run-time, flash storage, and the system keychain.  Research outcomes include an open source secure application framework, comprising an application container, developer, and validation tools/techniques.  With iMAS, a developer can leverage our research to considerably raise their iOS applications security level in a measured way.

Objectives
Our plan is to demonstrate the concept, partner with hReader.org, and make the iMAS source code available for community use through an open source license.  We will endeavor to create a community dedicated to addressing the need for increased security controls for iOS mobile platforms.

Activities
All activities on the iMAS project will be available through the open source. This includes access to the underlying source code, as well as planned security controls, forums, and blog or email list discussions.  iMAS is being distributed under the Apache 2.0 Open Source license.

Impact
We believe this project will raise the iOS applications security level. The external iMAS project website ishttps://github.com/project-imas. This project site includes information about the latest version of the software, plans and schedules for new security controls, vulnerabilities addressed, and access to the open source code.  It is also the site from which MITRE and iMAS partners can access and download the iMAS source code.

Public Release No: 13-0121

Problem
Patients’ health data is needed by physicians, other caregivers, laboratories, and researchers. Today, to share data selectively, the patient must fill out a consent form specific to each provider and often each recipient. Given the variability of provider forms and the patients’ need to thoroughly understand how their decision to share or restrict sensitive health data affects their care, patients have many details to comprehend and keep up to date. Consent management is a major barrier to data sharing today, and inadequate sharing can lead to suboptimal treatment, duplication of work, and slow research progress.

Objectives
A patient-centric consent management system can enable the patient to create one careful specification that is compatible across all their providers. The latest version would always be available from an Internet-accessible consent repository. We seek to enable the majority of patients to create appropriate consent policies that address the requirements of 50 states and multiple usage modes (e.g., emergencies).

Activities
We have four primary activities planned for FY13: -Explore dependencies of our research on legal regulatory policy, and recommend both technical and policy strategies. -Extend technical and policy work to manage uncertainty (e.g., patient consent to share medication lists, based on due diligence rather than absolute certainty in testing for revealing mental health issues). -Conduct knowledge exchange with our sponsor to enrich designs that will empower patients to make meaningful choices about data sharing tradeoffs, without special training. -Identify barriers to adoption and possible regulatory ameliorations.

Impact
Our research shows the feasibility of meaningful, patient-centric granular consent to control how health data is shared, addressing issues of providers, Health Information Exchanges, and privacy advocates. Our “uncertainty-aware” consents give patients greater transparency and control and may reduce providers’ liability concerns. We have already begun to transfer our ideas to our sponsor and to privacy advocates.

Public Release No: 13-0657

Problem
Clinical Quality Measures (CQMs) report artifacts that measure processes, experiences, and outcomes of patient care delivered by healthcare providers. Certain CQMs are being selected with the intent of supporting the Centers for Medicare and Medicaid Services (CMS) Pay For Performance (P4P) program. These CQMs will be used to align the compensation of providers to the overall quality of care of their patient population. Metrics that encourage the adoption of higher quality CQMs in P4P models are expected to measurably bend the healthcare cost curve.

Objectives
We plan to design an open source framework to analyze the complexity and feasibility of CQM calculation based on information extracted from the Health Quality Measures Format (HQMF) data standard. We will then enhance this framework to incorporate findings from both financial claims and compensation data. The final goal is to develop a unified software model to automate the assessment of CQMs based on complexity, financial claims cost, and data interoperability.

Activities
We will develop the Kamira software framework to assess the quality of CQMs, considering three dimensions: complexity, financial costs, and data availability. We will use open source processes to faciliatate both collaboration and the transition from a MITRE-supported research activity to a sponsor-championed project. To support outreach, we will generate a summary of the Meaningful Use Stage 2 CQMs complexity findings. A public web site will be available, along with the CQM complexity analysis open source software.

Impact
We presented our project goals and findings at the Office of the National Coordinator for Health IT/CMS January 2013 LEAN Kaizen Conference. We expect a CQM complexity model to directly benefit federal leaders driving the evolution of CQMs as part of Meaningful Use Stage 3 by helping to ensure the next generation of CQMs will be more tractable by bounding complexity. Kamira’s capabilities will have the added benefit of harmonizing the intent of measure stewards working to realize their clinical vision of CQMs with operational Electronic Health Record vendors who are limited in the data that they collect. Lastly, Kamira will provide visibility into the costs for providers associated with addressing CQM gaps.

Public Release No: 13-0444

Problem
We aim to improve the affordability, effectiveness, and efficiency of lower-limb, prosthetic rehabilitation by precisely quantifying, via motion capture, transitional (e.g., turning, changes in speed, gait) and nonmodal (e.g., active standing) movement, allowing clinicians to make quantitative assessments of recovery.  Repetitive recordings of complex movements in an unconstrained environment are difficult or impossible to obtain using current techniques, and are rarely studied despite their importance to recovery from injury.

Objectives
We are developing a suite of tools to programmatically guide amputees through automatically generated sequences of movement. These movements can be transitional, nonmodal, or impromptu.

Activities
Our system guides participants through transitional, nonmodal, and impromptu movements by projecting a laser dot onto the floor.  The dot is generated by a laser mounted to a computer-controlled pan/tilt mount and will trace curved paths (to encourage periodic, transitional, and nonmodal movement) and semicircular arcs around the participant (to encourage aperiodic, nonmodal movement). We will complete the building and tested of the system this year.

Impact
We will transition this system to the Gait Laboratory at Walter Reed National Military Medical Center to increase their data collection capabilities, allowing them to control a patient’s movement in an accurate, repeatable, and unique manner. This will, in turn, allow them to advance the state of the art in rehabilitative care.

Public Release No: 13-0540

Problem
MITRE has been facilitating collaboration both internally and with our partners via the use of social tools. While a number of these tools have had subjective success, it is very difficult to measure success objectively and link it to specific behaviors at the individual, community, or enterprise level. Because adoption and establishment of usage patterns is slow, the relationship between those usage patterns and changes in business processes takes time to emerge. In many cases, appropriate metrics for tying user behavior to project success do not exist. Most social metrics are designed for marketing; very few track the establishment and maintenance of relationships over time, allowing little assessment of the network effects that tend to be most valuable within a business context.

Objectives
The desired outcome is to use longitudinal evaluation techniques and visualization to show the individual, community, enterprise, and partner value of social tools for the enterprise.

Activities
We will conduct an assessment of longitudinal effects through the use of survey snapshots over time,  analysis of activity data, and visualizations to answer the following and more: -How are our external partners using Handshake? -Are there correlations between group types, behaviors, and success factors? -Does Handshake support the reader-to-leader framework? What motivates users to contribute? -Does Handshake help to bridge virtual gaps: organizations, locations, job level?

Impact
Organizations are faced with the difficulty of evaluating and verifying the value proposition of social tools. Our ability to do a detailed analysis, have access to our users, and control and understand many of the criteria impacting adoption, use models, and value puts us in a unique position to understand both how to assess value and recommend techniques to accelerate adoption.

Public Release No: 13-0690

Problem
Defense contractors develop their High Performance Embedded Computing (HPEC) software not just for vector and signal processing but also for infrastructure, such as processor/core allocation, management, job scheduling, fault detection, fault handling, and logging.  Since sensor and compute platforms are continually updating, often with different hardware, the old version of software is often discarded and new versions are redesigned and reimplemented. This is an inefficient use of developer time, as there exist numerous commercial and open-source frameworks that provide ready solutions to multi-processor infrastructure needs.  However, the HPEC community is reticent to use these as many of them are based on Java, which can be slow.  With intentional design for performance, this need not be the case and it should be possible to mix Java and C for enhanced productivity.

Objectives
Our idea is to use Java compute-grid frameworks mixed with C compute code and compute kernels for graphics processing units.  In this continuing work, our goal is to show that the performance of such a mixed HPEC application can be maintained and the final application will be robust in the face of compute-node failure.  We will do this in the context of an HPEC domain application.  We will also show that HPEC high-speed communications can be utilized from these frameworks.

Activities
We will analyze candidate Java infrastructure frameworks to compare their capabilities and characterize select ones based on key metrics such as throughput, number of tasks per second, and node-to-node latency.  One such framework will then be used to implement a domain application that is already implemented in C.  Then we will compare the two versions of the application with regard to robustness, size, and ability to meet compute deadlines.

Impact
Last year we showed that Java could perform comparably to C on individual compute nodes given an appropriate design for performance.  The expected results of this year's work will fill in the questions of performance in a multi-node application.  This opens the door to mixing Java and C (including optimized compute libraries and graphics processing units).  This allows contractors to implement functionality more quickly and have an easier time maintaining and upgrading it.  Solutions should be more robust, some benefitting from extensive open-source testing.  Because of our work, projects will have a greater pool of tools for design, profiling, load testing, unit testing, and dependency analysis.

Public Release No: 12-4846

Problem
The U.S. government has a strong incentive to control medical costs, especially those of Medicare, Medicaid, and the Department of Veterans Affairs (VA). Detecting medical conditions early (or, even better, preventing them) can keep such conditions from becoming more serious and costly.  Our project team is using patient care data to develop systems that either detect or predict a condition for some patients, leading to changes in those patients’ treatment.

Objectives
The MIMIC2 intensive care unit (ICU) data set is a rich data set that includes both waveform and non-waveform data. Such data presents the opportunity to combine various types of information, such as respiration waveforms, with more traditional structured data, such as medications given, to detect and/or predict when interventions may help.  An initial example task is to predict when an ICU patient will need invasive ventilation at some point during an ICU stay.

Activities
We are extracting a subset of MIMIC2 data useful for predicting future invasive ventilation. A system using just waveform data will be developed by looking at a respiration waveform, using such measures as the respiration rate variability. A separate system restricted to using only non-waveform data will be developed using SVMs, a machine-learning technique.  Finally, we will develop a system combining both types of analysis.

Impact
There will be two levels of results and impacts. More specifically, an earlier or better prediction of needing future invasive ventilation may prompt clinicians to either take steps to try to avoid the need for invasive ventilation or to skip some intermediate steps that may be taken, like non-invasive ventilation, and proceed straight to invasive ventilation. More generally, a successful system will show (disseminate via publication of results) how one might combine waveform and non-waveform clinical data to make better and earlier health issue detections or predictions.

Public Release No: 13-0189

Problem
It is estimated that 62 percent of the U.S. federal tax gap (about $450B annually) is the result of underreported individual income hidden within innovative tax shelters. These shelters consist of a complex web of interacting tax entities, such as corporations, trusts, partnerships, and individuals, that  evolve over time. Consequently, tax enforcement strategies that are successful in one year may not be successful in another.

Objectives
We are investigating tax evasion schemes that fall into the category of Artificial Basis Step-up Transactions (ABST) as the baseline for our study. In these schemes, the value of an asset is artificially increased such that its future sale results in a capital loss, which is then offset against a capital gain elsewhere to minimize overall tax liability. Our hypothesis is that a Genetic Algorithm-based framework can be used as a tool to identify and evolve ABST-based tax shelter schemes.

Activities
To test our hypothesis, we will generate the known "Son of Boss" ABST tax shelter scheme by evolving a population of tax entities and their corresponding asset transfers over time. The Son of Boss scheme has exploited the IRS definition of "contingent liabilities" and is estimated to have resulted in $6B of understated tax.

Impact
This research could reduce the IRS's workload by providing a set of frequently occurring asset transfer patterns that could be flagged as potentially tax evasive. In addition, the IRS could deploy maximum deterrence efforts with more timely effectiveness to improve voluntary tax compliance.

Public Release No: 13-0250

Problem
Over the next nine years, the Federal Aviation Administration (FAA) plans to hire 12,000 new controllers, while at the same time introducing the Next Generation Air Transportation System (NextGen).  Current training programs may not adequately address NextGen’s changing technology and philosophy.  There is a need to bridge the training gap between current operations and NextGen, and to teach the theory of Traffic Flow Management (TFM) and Collaborative Decision Making.

Objectives
We are developing a theory-based curriculum that includes TFM ideas and concepts, system integration, crew resource management, and decision models.  We will produce a TFM textbook and lab environment to support the curriculum, as neither currently exists. This approach will be taught as an experimental class beginning in the Fall 2013 at several universities.  After a successful experiment period, this class will be transferred to the colleges and offered on a permanent basis.

Activities
We have executed partnership agreements with two training schools. The first prototype course will be offered in the Fall semester of 2013. Students will be tracked throughout the class and throughout their career with the FAA through evaluations.  These evaluations should support our theory that by taking this class students will have a shorter learning curve on new technology and a better overall understanding of the National Airspace System.

Impact
Our research, if successful, will establish new curriculum requirements for the training standards schools use to prepare the future air traffic work force. It will also provide an opportunity to expand the capabilities of the FAA’s current education process to employ integrated solutions, lab exercises, and an information-rich approach to system/domain connectivity.

Public Release No: 13-0410

Problem
Determining the safety implications of potential NextGen operations is crucial for the Federal Aviation Administration as it adopts new operational concepts, while maintaining or improving safety.  Measuring these changes in safety holistically is difficult with available analytic platforms. Many safety models are individualized models that cannot quantify the aggregate risk associated with a prospective operation since they are generally tailored to answer only a specific safety question.

Objectives
We will develop a unified simulation model that uses empirically derived trajectories, capturing the variability of flights, to quantify the change in safety from a prospective operation.  Each simulation agent (aircraft, pilot, controller) in the model contains models of variation in behavior, both internally to the agent and in its interactions with other agents.  Analysts can add or extend functionality for their project, collectively increasing the fidelity simulation environment over time available to the entire user community.

Activities
In the first stage, we are constructing base agent objects: pilot, controller, radio, and aircraft.  A trajectory object with realistic variability will be constructed from straight and turn segments from historic data.  Our initial goal is to measure proximity collision risk for pairs of hypothetical NextGen procedures. Going forward, we will increase the complexity of the base agents and incorporate MITRE 3D, a GIS framework.

Impact
The final product will be be a common modeling and simulation platform for safety analysis. This agent-based simulation platform will allow for analysis of multiple safety impacts from a given operational change. The system is not limited to procedural changes; we can also examine fatigue, workload, and decision making under uncertainy. Examining aggregated operational factors in a single simulation will greatly increase our understanding of the holistic change in risk when instituting a new concept of operation. This capability will permit NextGen decision makers to better incorporate safety risks in their deliberations.

Public Release No: 13-0898

Problem
Emerging memory technologies based on memristive nano-architectures are now widely accepted as the future replacement for CMOS based memory technology. The problem is that a strong fundamental circuit theory supporting how to optimally use these novel devices appears to be missing since the field has just been born. The storage or communication of sensor data using bits as opposed to analog code words has been shown to be strongly suboptimal from an information theoretic point of view. To achieve optimal analog memory utilization we need an information-theoretic approach to designing an analog memory controller that is matched to the device physics of memristive devices and architectures.

Objectives
Our idea is to apply analog error correction codes to provide noise-immune storage of numbers and implement the encoder and decoder maps via computation with dynamical memristive systems.  The analog encoder takes the form of a Shannon-Kotelnikov map, and nonlinear feedback control with memristive devices is used to drive the internal memristive device state variables to the desired analog code words—thus bridging nonlinear computation, coding, and control theory into a single framework with minimal power and integrated circuit area.  Implementing the memory controller using networks of memristive devices avoids the Von Neumann bottleneck and allows massively parallel read/write operations.

Activities
We have developed a novel analog code construction technique based on the notion of densely packing hyper-tubes in a high dimensional Euclidean space using differential geometry of curves and concepts from knot theory. We have also identified and are continuing to explore analog computational models that can perform the needed encoder/decoder functions and can be implemented with memristive circuits. We have begun and will continue to explore the design and simulation of memristive circuits in SPICE and the proper method for feedback control within a memristor-only based circuit. We continue to to submit papers and patents for this novel analog memory controller framework. 

Impact
A successful analog memory controller will have significant advantages over a digital memory controller in terms of power, size, and cost.

Public Release No: 13-0304

Problem
Methods exist for the exploitation of structured, cooperative, full-motion-video (FMV) streams. These techniques enable image mosaicing, georegistration, 3D reconstruction, and change detection from images of a 3D scene. These methods all rely on some level of cooperation with the image sensor. Noncooperative video sources--such as cell phone video, surveillance video, and even foreign news reports and archived friendly UAV video--generally cannot be exploited by tools developed for cooperative FMV streams. The main limiting factor for metric exploitation of uncooperative video is the lack of an intrinsic optical sensor model for the optics.

Objectives
We plan to derive and implement a series of novel, data-driven machine learning techniques for 3D exploitation of arbitrary non-cooperative FMV sources. Our primary deliverable will be a tool that analyzes the projective geometric constraints throughout a video sequence and simultaneously solves for the intrinsic sensor model and extrinsic sensor pose with respect to a 3D reconstructed scene. Such a tool will provide automated metric access to the massive archive of intelligence data queued for manually reconstructed by expert operators.

Activities
We are constructing a geometric “pose graph” describing the acausal interconnectivity of video frames within a collection. This graph may contain images from multiple video feeds and still image sequences. Next we'll generate a projective geometric feature vector to describe each edge of the pose graph. A machine learning algorithm describes the edge in terms of a gross description of the camera motion (e.g., rotation only, general motion, or scene depth relief). A reconstruction agent then applies rule-based logic to develop a traversal trajectory for correct metric reconstruction of the video sequence.

Impact
Our team has constructed a full FMV simulator capable of generated dynamic zooms and arbitrary motions (including pure rotations). We have developed a set of 8 distinct floating point and binary entries that form the basis of the novel projective geometric feature vector.

Public Release No: 13-0744

Problem
Many systems used by governments, militaries, commercial enterprises, and individuals require radio freqency (RF) spectrum. Demand for spectrum is increasing but the supply is finite. Thus, meeting the demand requires more agile spectrum management approaches.

Objectives
The purpose of this project is to improve spectrum management and increase spectrum reuse.  Model-Based Spectrum Management (MBSM) is based on the creation and exchange of Spectrum Consumption Models (SCMs), which are machine readable data that capture the temporal, spatial, spectral, and behavioral boundaries of spectrum use. They serve as the loose coupler of dynamic spectrum management and access systems.  Attendant computations arbitrate the compatibility of modeled uses.

Activities
We are modeling RF systems and verifying that the modeling constructs algorithms are effective. Then we will revise the modeling constructs based on this experience and update the modeling manual with these revisions. Next, we will shepherd modeling through the standardization processes of the IEEE Dynamic Spectrum Access Networks Standards Committee and promote MBSM as the solution to the Spectrum Access System proposed by the President’s Council of Advisors on Science and Technology in their 2012 report “Realizing the Full Potential of Government-Held Spectrum to Spur Economic Growth.”

Impact
If fully implemented, MBSM will fundamentally change spectrum use. MBSM and SCM enhance spectrum management in many ways: -In regulation, SCMs define a user’s spectrum usage rights -In commerce, SCMs capture the quanta of spectrum that are traded  -In technology, SCMs convey assignments and policy to RF systems -In operations, SCMs enable dynamic and flexible management.

Public Release No: 13-0710

Problem
Enterprise-level simulation models of government program performance are in high demand by MITRE’s sponsors. These models, however, are often time consuming to develop, test, and deploy for analysis.  How can MITRE get these capabilities to sponsors more rapidly and with greater consistancy?

Objectives
Our goal is to develop foundational, generic modeling components and modeling processes so that modelers across MITRE can more effectively develop models that can be used with sponsors for large impact.  This will support reusability and commonality across research efforts and support a rapid, extendable modeling platform that will enable transformative decision making As we learn more about the unique modeling requirements for the government services ecosystem, we will develop tailored components.

Activities
First, we are developing common modules for performance, cost, and schedule for simulation models, as these are common features  of many civilian agency models. We will evaluate the execution of models in different environments using high-performance computing clusters and develop best- practice templates and guidance for future models. We will also develop a custom interface to make deploying budget-linked performance experiments easier to configure and run for non-experts.

Impact
With reusable model components and tools, both generic and tailored, MITRE modelers will be able to quickly and reliably create simulations for customers by focusing on unique aspects of a model and not on common functionality. We will be able to quickly develop models to evaluate new strategies, demand, allocations, cost impact, and more. By reusing components, models can use best-of-class features, and sponsors will only need to learn an interface once. We have already shared some of our tools with direct work projects to communicate and share performance and schedule data.

Public Release No: 13-0353

Problem
Modern wireless communication systems must be capable of overcoming channel imperfections attributed to fading and interference. Failure to compensate for non-ideal channel conditions can prevent users from developing innovative applications, which can hinder mission effectiveness. The fundamental problem with many of the emerging wireless communication systems for wideband tactical use is the lack of emphasis on improved resiliency and robustness. Instead, many of these systems have focused on improving network services, such as throughput.

Objectives
The design of reliable and robust communication signals and receivers has been the genesis behind development of advanced commercial cellular and wireless local area networks. Our research project intends to adapt and apply mature technologies from commercial wireless standards for tactical applications. Some of these technologies include: Multiple Input Multiple Output, Space Time Coding, Multiple Carrier Modulation, and schemes for Multiple Access. By leveraging commercial developments, we intend to design a scalable waveform to fit emerging needs across a broad user base.

Activities
Ongoing efforts include waveform and algorithm development to improve performance beyond that of existing wideband tactical waveform standards. We are evaluating our approaches using standardized channel models to ensure convergence on a practical and robust solution. An important part of our work is focused on enabling link adaptation algorithms for additional performance gains and ease of use. We also plan to develop two baseband prototype transceivers and characterize their performance under controlled conditions.

Impact
To date we have generated results that characterize and verify this new waveform design under various channel conditions. We are in the process of developing key components such as synchronization and equalization. We have devised approaches for making this waveform flexible to cover a broad range of applications. The performance of this new waveform appears promising and, if successful, it can enable longer range tactical communication systems, higher throughputs in channels subject to fading and/or interference, and an overall increase in reliability. A system implementing this waveform could empower a multitude of new applications for tactical users.

Public Release No: 13-0232

Problem
There is a movement to migrate analytic work to the data cloud  to reach a consistent method for accessing and processing data that can scale to large volumes. Techniques are needed to manage workflows that include next-generation cloud-based analytics and analytics that have not been transitioned to the cloud.

Objectives
MOSAIC is a MITRE-developed Human Language Technology architecture for workflows of discrete analytics. MOSAIC CBX (Cloud Based Extension) migrates the MOSAIC architecture into the cloud in concert with current prevailing technology directions. We apply these technologies to MOSAIC’s core capabilities: data bus, adapters, analytics, execution, and workflow.

Activities
We are evaluating data cloud alternatives for MOSAIC’s common interchange model, adapting it to lessons learned in prior work on BigTable-based triple-stores, and considering alternate solutions like Big Data. We will assess formats, layouts, and indexing methods for preserving the rich representation, efficient ingest, and querying and reasoning capability. Subsequently, MOSAIC CBX will be extended to include both cloud and non-cloud analytics. We will examine alternatives for the MOSAIC executive that allow distributed execution of non-cloud analytics (ideal for analytics that cannot be reengineered), effectively creating a grid solution. We will also evaluate using MapReduce for workflow.

Impact
MOSAIC CBX will provide adaptable means for migrating analytic workflows to the data cloud, allowing for continual capability during the transition.

Public Release No: 12-4374

Problem
Via advanced simulations and laboratory experimentation, this project is attempting to realize much smaller, lighter, and more power-efficient C4ISR systems by developing, harnessing, and integrating nano-enabled materials, circuits, sensors, and energy storage components.

Objectives
Our team is collaborating with Harvard University to design, fabricate, and demonstrate both 1x2 (2-tile) and 4x4 (16-tile) nanoprocessors, building upon its previous success in developing the world’s first nanoprocessor, a 1-tile system.  Also, the team is pursuing unique approaches to speeding the simulation of nanomaterials, as well as of nanocircuits.

Activities
The team plans to complete the above-mentioned 1x2 nanoprocessor and write a technical article about that, while beginning work on the 4x4 system, with a goal of completing it by the end of FY13.  Further,  the team is advancing the development of a first-principles multiscale materials modeling method.  In addition, the team will conduct the renowned MITRE Nanotechnology Summer Student R&D Program for the 21st year.  Student staff and their adult mentors will investigate the topics above, in addition to nose-like nanosensing, nanoforensic tags, and smaller, lighter, nano-enabled portable power systems for soldiers.

Impact
In the past two years, our project team had much success in nanoprocessor development. Developing the 4x4 nanoprocessor this year could help enable as much as a ten-fold reduction in the size and power requirements for government electronic systems.  Over the past seven years, in its multiscale modeling R&D, the project team has discovered new laws of physics that govern the behavior of materials from the nanoscale to the macroscale. If we can use those laws to develop a first-principles multiscale modeling method, it could greatly speed the simulation of materials with novel properties, perhaps enabling materials by design.  Results from this project over the years have been instrumental in serving a number of MITRE’s sponsors. 

Public Release No: 13-0583

Problem
Environmental concerns related to noise, air quality, and the climate change impacts of aviation activity present significant constraints to future aviation growth. Decision makers will increasingly seek out information on the system-level environmental implications of proposed changes to the air transportation system in addition to measures of operational performance. Our research is aimed at bridging the gap between air traffic simulation tools and environmental models to enable a more comprehensive assessment of these tradeoffs.

Objectives
Our work focuses on linking MITRE's systemwideModeler with the Federal Aviation Administration’s Aviation Environmental Design Tool (AEDT) to enable estimation of fuel burn, emissions, and noise impacts at the National Airspace System (NAS)-wide scale. Analysis capabilities developed through this project will be used in examining tradeoffs among operational and environmental objectives for select NAS-wide operational improvements and/or policy measures.

Activities
This year the team is focused on maturing the analytical interface between the systemwideModeler and AEDT. Our primary activities will include research on appropriate case study topics, case study formulation and modeling, analysis of systemwideModeler and AEDT outputs, reporting of study findings, and methodology modifications based on lessons learned through case studies.

Impact
By enabling a system-wide assessment of tradeoffs between operational and environmental performance, this work will benefit a wide range of aviation stakeholders.  Analysis capabilities developed through this reserach will expand the NextGen benefits metrics portfolio to include environmental metrics.

Public Release No: 13-0422

Problem
Cyber threat reports and analyses are written in unstructured format or "free text." Analysts use wikis, blogs, and forums in the course of analysis and collaboration, but they must then re-enter information such as IP addresses, malware artifacts, and URLs by hand into a variety of structured formats to configure defenses or populate threat databases.

Objectives
This team is exploring the use of Natural Language Processing (NLP) technologies to automate the extraction of cyber threat information from unstructured text sources.

Activities
Using a team of cyber analysts and NLP experts, we will employ MITRE-developed technologies such as the jCarafe trainable NLP toolkit, Callisto, and the MITRE Annotation Toolkit, to study different tasks and data sets common to cyber analysis operations. We will leverage emerging cyber threat information standards such as CyBox ( Cyber Observable Expressions) and STIX (Structured Threat Information Exchange) to inform our work. We will focus on the extraction of context, (e.g., that a given host is used for command and control or to deliver exploit code) rather than merely the entities (IP addresses, URLs, hostnames), which is critical to proper handling and response to a given threat report.

Impact
If successful, this project will enable the integration and automated sharing of large amounts of cyber threat data among different defender communities. This would also reduce the tedious workload of analysts and support various approaches to automating analysis and data mining to discover new attacks. We can envision an automated process that could extract threat indicators 24/7, including context, and take defensive measures, such as blocking IPs or sinkholing DNS domains with faster-than-human response.

Public Release No: 12-5156

Problem
There exists a need for a sense-and-avoid capability for Unmanned Aircraft Systems (UASs) that is robust in responding to interruption of the command and control communication (C2) link and can perform against non-cooperative platforms. An aircraft-mounted solution is strongly desired so that in the event of a lost C2 link the UAS will have a self-contained sense-and-avoid capability.

Objectives
Recent advances in radar exploitation make it feasible to consider new bistatic radar technologies to solve this problem; the new technologies include very small antennas that have attractive size, weight, and power (SWAP) characteristics, as they rely heavily on digital signal processing techniques. This capability would provide the UAS with an organic capability to develop a picture of the local airspace sufficient to sense and avoid possible conflicts.

Activities
We are exploring two different signal processing approaches to support this effort, each having different hardware requirements and surveillance performance levels. Current research is focused on the development of the different tracking approaches and assessment of the expected performance of each approach in an end-to-end context. Next we will collect flight data to further assess the techniques, validate the prototype signal processing algorithms, and facilitate the development of the processing chain.

Impact
Our research could lead to a cost-effective on-board sensor that provides UASs with a sense-and-avoid capability for non-cooperative targets. 

Public Release No: 13-0777

Problem
The world in which we do systems engineering (SE) is complex, with many interacting parties that have both shared and conflicting interests. The current set of acquisition SE tools are optimized for large, “big bang” delivery; however, it is becoming clearer to our sponsor base that fiscal constraints will be driving down the number of big systems in development. In our immediate future, systems engineers in the acquisition environment are faced with an environment in which systems are flexibly architected to be incrementally produced or improved as needs arise. Systems will be articulated by technically savvy users, using rapidly changing technology options from a large number of stakeholders, via a mix of synchronous and asynchronous, distributed and co-located, engineering events. The question is: “How do we transform our SE approaches to be more effective in this expected, rapidly changing acquisition ecosystem?”

Objectives
We propose the creation of a scenario-driven, experiential environment within which system evolution concepts can be experimented with and matured for flexibly architected, distributed systems. We hypothesize that this will enable systems engineers to obtain appropriate exposure to and experience with complex stakeholder environments. This will consequently reduce information risk and associated engineering debt.  We are using a game as our delivery mechanism because prior research has shown that game play helps to elicit the creative problem solving that we believe is needed to excel in SE of the future. The immersive and experiential environment we create will better prepare systems engineers than traditional training mechnisms, which can only reinforce efficient behavior against known challenges and solutions. Our framework will enable experimentation and learning by systems engineers, and it will also support the capture of experiential knowledge, which can help inform current and future systems engineering research.

Activities
Using this framework, we will deploy game modules that allow for experimentation and training of systems engineers in the emerging systems engineering techniques.  We have begun to demonstrate how this experiential framework might support socially based constructs, such as the Systems Engineering to the Edge (SEETM) concept, including the multiparty engineering (MPE) paradigm. We will later show how more experimental co-engineering concepts might work. Our research sequence is as follows: -Create a SEETM scenario that includes newly emerging and previous SE research portfolio conclusions for social-based engineering regimes, such as MPE constructs. -Select a gaming framework that allows for rapid game module deployment, distributed multiplayer web-based immersion, collection of experimentation and learner actions, and analysis of and search for patterns. -Create a mini game module based on an aspect of the SEETM scenario articulated above -Demonstrate the prototype environment and mini game, and garner feedback from internal and external SE stakeholders to shape near-, mid- and long-term goals for future development of the framework.

Impact
The value of our approach is that the framework can help SEETM ecosystem architects define organizational, process, incentive and governance constructs, and it will immerse SE workforce agents in futuristic SE situations, at low technical and financial risk.

Public Release No: 12-4571

Problem
Government sponsors involved in Critical Infrastructure Protection (CIP) are increasingly required to produce sector-wide risk assessments to guide prioritization, resource allocation, and decisions.  However, no guidance is available for these specific homeland security assessments, leading to diverse, incompatible, and inconsistent results that cannot be rolled up for effective enterprise- or sector-level decision-making.

Objectives
We will develop and provide design guidance derived from analyses of actions that led to the success or failure of CIP risk assessment practices (techniques, tools and methodologies), and based on proven systems engineering principles.

Activities
We will: map legislative and regulatory boundaries constraining government agencies’ risk assessment practices; identify past and present risk assessment practices (using MITRE’s systems engineering expertise to support the design, implementation, and modification of practices); and incorporate risk expertise found across government, industry and academia into design guidance for CIP risk assessment practices. Then we plan to pilot application of our design guidance for CIP risk assessment practices with a sponsor.

Impact
Government sponsors involved in CIP who apply this design guidance will achieve risk assessment practices that are more effective in guiding prioritization, resource allocation, and decisions, and will be consistent across each organization and sector, enabling a larger roll up supporting decisions at all levels.

Public Release No: 13-0701

Problem
MITRE’s Integration Demonstration and Experimentation for Aeronautics (IDEA) laboratory is a real-time distributed simulation environment for human-in-the-loop simulation and visualization of air traffic management concepts. To expand our capabilities, we need to expand the lab beyond the physical confines of MITRE’s facilities; this would give us, for example, the ability to exploit reliable, high-performance computer resources on an ad-hoc basis.  The IDEA lab needs a smart way to scale “up and out.”

Objectives
In our research, we will apply current and evolving cloud computing technologies, such as Amazon Web Services, to extend the IDEA lab architecture. The objective is to make NextGen tools and experiments available without regard to constraints such as physical location, space, and computing and storage resources, as well as to control, deploy, access, and test typical IDEA lab simulations and demonstrations using the latest industry cloud technology.

Activities
We will initially focus on a virtual private cloud infrastructure, based on Amazon’s Elastic Compute Cloud technology, that adheres to information security needs. This work will be followed by measurement activities to evaluate and improve simulation usability and to enable secure remote access.  Next we will develop a concept of operations for cloud-based, real-time simulation and document the results, lessons learned,  and best practices from this research.

Impact
If this effort is successful, MITRE and its partners will have an unburdened, clear, and automated way to execute, share, and manage simulations and demonstrations in a cloud-based infrastructure. This would vastly improve the scalability of the lab architecture, allowing for a theoretically unlimited number of simultaneous simulation events. Experiments could be as large as the software architecture would support and would require little effort to procure and maintain necessary computing and storage resources.

Public Release No: 13-0761

Problem
This research investigates the problem of extracting relevant information from multiple data sources and using it to make inferences of the underlying activities.  Manual exploitation of data sources is time consuming and is not feasible with today’s collection rates.  Automated systems have been developed to address this processing need but their output contains errors.  Our goal is to take the output of these automated systems, which contain uncertainty, and use statistical inference to discover the underlying activities that cannot be directly measured.

Objectives
The desired outcomes are to achieve: -Automated, probabilistic determination of places of interest.  -Aggregation of attributes at these locations over time.  Automated detection of kinematic patterns of interest.  Probabilistic asssignment of kinematic patterns to locations.  Machine learning over both attributes and kinematic patterns to discover combinations of features that can be used as indicators of interest.

Activities
The process of inferring places of interest has the consequence of also returning spurious locations.  This additional noise should decrease with more persistant data.  Controlled experiments will be performed using synthetic data with known, injected signals to determine how much persistence is required to increase the signal-to-noise ratio to acceptable levels.  Kinematic patterns en-route between locations will be simulated to develop mehtods of discovery.  These kinematic activities will then be probabilistically associated with locations.  The developed techniques will be tested on real-world data.

Impact
If successful, Nexus will add considerable utility to those exploiting Wide Area Motion Imagery data. Currently, the majority of data goes unused. Nexus provides an automated way of extracting and distilling useful information from the data for an analyst to review.  It also fuses in any other form of data as long as it is tagged with spatio-temporal information.

Public Release No: 13-0607

Problem
Relative to an out-the-window view on a clear day, a considerable amount of information is lost in airport surface surveillance systems. Thus, current surface surveillance systems are not permitted to be used as a primary source of information. This situation results in a considerable reduction in capacity at times when there is low visibility and surface surveillance is more heavily relied on.

Objectives
To restore information lost with surface surveillance, this research will leverage a currently untapped human information processing channel: non-speech audio. We believe non-speech audio could provide improved continuous status information by using naturalistic aircraft sounds to communicate qualitative information about aircraft state on runways.

Activities
We began our research by conducting a needs analysis, identifying what information the sounds need to communicate, and a benefits analysis, estimating the magnitude of capacity benefits.  We will develop a non-speech audio prototype and conduct a human-in-the loop simulation to test the concept. We will collect both objective and subjective data  from the controllers about their experiences. If the initial exploration effort is succesful, then further feasibility, human performance, and benefits analysis would be warranted.

Impact
A deployed non-speech audio for runway status system has the potential to increase airport throughput during weather that limits out-the-window visibility, without sacrificing safety.

Public Release No: 13-0780

Problem
Forensic facial analysts compare the anatomical features they observe in images of unknown individuals to those they observe in comparison images of known individuals. Typically, these comparison images are retrieved from large databases using facial recognition technologies that automatically search for images of faces similar to that of the unknown individual. Based on the anatomical similarities and dissimilarities they observe between images, facial  analysts then estimate the likelihood that an unknown individual is the same person as a known individual.  Forensic facial analysis bears an obvious resemblance to the kinds of face processing we perform in our everyday lives. As a result, the scientific community has suggested that forensic facial analysis may be subject to the same limitations as naturalistic face perception (e.g., reduced accuracy at perceiving faces of individuals with racial/ethnic backgrounds different from one’s own). However, these claims have not been evaluated empirically, and little or no research has investigated the nature of forensic analysis and its relationship to naturalistic perception.

Objectives
We will conduct a study that directly compares forensic facial analysis with naturalistic facial perception. The core task will require participants to determine whether or not pairs of images depict the same person. Images will contain faces of people with either the same or a different racial/ethnic background from the participant. A subset of the participants will perform additional forensic analyses before indicating whether or not images depict the same person. We hypothesize that participants in the “forensic” condition will be more accurate at same/different judgments than their counterparts in the “control” condition, and we further hypothesize that forensic participants will be less affected by whether or not images depict individuals with the same racial/ethnic background as their own.  Our primary purpose in having forensic participants perform forensic analyses is to determine how forensic analysis influences their ability to determine whether images depict the same person. Our secondary purpose is to collect information about correlations between different facial features evaluated during forensic analysis. Using the data, we will build and evaluate regression models that predict whether or not two images depict the same face based on perceived similarities/dissimilarities between individual facial features.

Activities
We have developed an initial experimental design and have begun evaluating sets of images for use as stimuli. Moreover, we have begun exploring different platforms for conducting experiments and different statistical procedures for analyzing data, including various approaches for aggregating forensic analysis data using optimal scaling. Our next steps are to assemble experiment stimuli, develop experimental delivery software, and conduct an initial version of the study with MITRE participants. We will then develop and deploy a modified version of the experiment on Amazon Mechanical Turk, a crowdsourcing platform that provides an inexpensive means of collecting data from a large number of participants. The Mechanical Turk experiment will focus on forensic analysis and will provide the bulk of the data for developing and testing regression models.

Impact
Results will benefit MITRE sponsors in the forensic facial analysis community, particularly those involved in developing facial analysis training, tradecraft, and analytic standards. An increased understanding of human facial analysis may also inform the development of new automated facial recognition technologies.

Public Release No: 12-4967

Problem
Obesity accounts for 21 percent of U.S. healthcare costs and obesity rates are climbing at an alarming pace. Countless initiatives have been mounted to combat this epidemic, with billions of dollars spent on prevention. Programs are targeted at specific populations, such as children, Medicare recipients, minorities, or individual cities and counties. Program evaluation relies largely on annual surveys, such as the Centers for Disease Control and Prevention’s (CDC) Behavioral Risk Factor Surveillance System, results of which may not be timely for program needs or match the precise target demographic.

Objectives
Our Persistent Health Assessment Tools (PHAT) will equip public health policy makers with more precise tools and more timely information for measuring the success of obesity prevention initiatives. We will use social media to supplement traditional surveillance by making real-time measurements to infer behavior among Americans. Results will be presented in a way that enables policy makers to observe the populations of most interest to them.

Activities
We will model recent American obesity trends using CDC surveys as reference data, and infer the gender, age, and location of social media users using techniques previously developed at MITRE. Then we will combine these data sources to identify and track social media predictors of real-world behavioral trends that impact obesity rates among populations of interest. While social media users typically won’t post explicitly about the outcomes of interest to policy makers, our algorithms may find value in tweets like these: -Beverly Pruden @beverlypruden:  Starting week 4 Couch to 5K tomorrow, looking forward to it for first time, see you on Wilkens! #couchto5K #running -Dex @dex1138:  I just ousted @tim02358 as the mayor of KFC – Kentucky Fried Chicken on @foursquare! 4sq.com/p8Sjc7 We will isolate tweets of interest and import them into an interactive dashboard where policy makers can browse, cluster, and search through messages to observe specific instances in which a program's outreach was successful and identify ways that outreach could be improved.

Impact
Timely projections of obesity rates for tailored demographic groups will enable public health stakeholders to make faster decisions about the success of their obesity-prevention programs, and allocate their resources more efficiently.

Public Release No: 12-4453

Problem
The Department of Homeland Defense (DHS) and Department of Health & Human Services (HHS) are independently developing data standards that shape the healthcare enterprise. “Pre hospital” data standards are being developed by DHS, while “In-hospital” data standards are being developed by HHS. The respective standards development organizations are the Organization for the Advancement of Structured Information Standards (OASIS) and Health Level Seven International (HL7). Because these sets of standards are being developed independently, a patient’s electronic data history stops and starts at the “ER door.” This lack of interoperability is potentially lethal to a patient. Electronic information should be able to flow seamlessly back and forth across the “ER door.”

Objectives
MITRE operates both the DHS and HHS FFRDCs and is engaged in both OASIS and HL7. Leveraging these engagements, we will identify appropriate data standards that can be harmonized, permitting inoperability and the exchange of electronic data. Additionally, we will identify areas of opportunity to increase engagement in the standards development organizations and provide guidance/recommendations on that engagement. Finally, we will provide storyboards of the relevant use cases and provide a reference implementation with appropriate documentation.

Activities
We will evaluate the OASIS Tracking of Emergency Patients (TEP) and Hospital Availability Exchange (HAVE) 2.0 data standards in conjunction with the HL7 Reference Information Model (RIM). The application of the OASIS Emergency Data Exchange Language (EDXL) Distribution Element (DE) as a message routing mechanism for HL7 messages will be explored. We will also identify specific use cases/workflow for data exchange. Finally, we will develop a lab-based reference implementation for data transformation and exchange with a reference implementation guide.

Impact
This work will show that seamless exchange of information across the “ER door” is possible and will support cross-agency engagement. This work will also lay the foundation for future work harmonizing data between the OASIS and HL7 data standards, allowing for improved information exchanges between various DHS and HHS organizations. Ultimately, this work will lead to improvement in the “continuum of care” for emergency patients.

Public Release No: 13-0617

Problem
The U.S. power system's increasing demand, lack of supply margin, aging infrastructure, and difficult capital and regulatory environment create the perfect storm for increasing wide-scale and persistent outages. The probability of wide-scale disruptions is compounded by natural disasters, solar geomagnetic disturbances, and terrorist threats. While interconnectivity of the grid provides redundancy, it also provides a path for transient events to propagate. Disturbances, such as those of August 14, 2003, and September 8, 2011, accentuate the continuing need for new technology that can warn operators in real time when a power system approaches critical operating points.  

Objectives
The progression of a power transmission network from a stable operating state to one that could result in degraded network performance has been extensively reported in the literature. There is a long history of using eigenvalue analysis to evaluate these types of critical transitions in control theory. Extensive research shows that the eigenvalues of the linearized system equations can be used to predict proximity to voltage collapse and small-signal instability. However, accurately estimating eigenvalue (or mode) trajectories in a large system requires accurate models and large quantities of high temporal resolution sensor data. Short transients, such as from noisy loads or variable sources, can affect system dynamics in ways that are not captured by eigenvalue analysis methods based on relatively slow synchrophasor data. A particularly interesting technique exploits the observation that complex systems show evidence of “critical slowing down” before they reach points of critical transition to instability.

Activities
We will develop a time history of disturbances across the continential U.S. (CONUS), which will be derived from disturbances catalogued by the University of Tennessee, Knoxville, and NOAA, as well as automated reporting by local power companies. That database of disturbances will be compared to high sample rate recordings of the full voltage and current waveforms collected by MITRE at nine sites across the CONUS. We will then develop a method that identifies major disturbances in the sparse high sample rate data set. We will evaluate the application of various stability metrics to diagnose topologic and temporal trends toward instability. Indicators toward instability could be used for event detection and eventually to trigger real-time interventions, including islanding or automatic load shedding.

Impact
We hope to show that high sample rate collection of the voltage waveform at various locations can be used to detect disturbances across a wide geographic area. Such a tool would be of value to the regional power grid managers, the North American Electric Reliability Corporation Electricity Sector Information and Analysis Center, the Department of Energy, as the Energy Sector Specific Agency, and the Department of Homeland Security, as the overall CIP coordinator and participant in the Electric Sector-Government Coordinating Council.

Public Release No: 13-1062

Problem
The nation's Smart Grid and legacy power grid upgrades depend on synchrophasor measurement units (PMUs) to achieve high reliability. These PMUs use time provided by Global Positioning System (GPS) units that can be jammed/spoofed, which could possibly lead to cascading faults and large-scale, long-duration blackouts.

Objectives
To mitigate GPS spoofing, both a GPS spoofing detector and a holdover oscillator with low long-term drift are needed. The GPS spoofing detector is the subject of other research. This team is focusing on identifying a holdover oscillator that can provide the holdover time when GPS is not available. Crystal, Cesium, Chip-Scale Atomic Clock (CSAC) and low-cost Rubidium (Rb) oscillators can provide holdover time. We will develop a modeling tool that incorporates the characteristics of CSAC and Rb oscillator drift, including magnetic field, barometric pressure, temperature, and intrinsic drift rate over time. The model can also be used to determine the maximum elapsed period after GPS is jammed until the PMU timing accuracy is exceeded.The model will also support analysis in other GPS-dependent critical infrastructure sectors (e.g., telecommunications).

Activities
Our plan is to obtain CSAC and Rb oscillator characteristics, identify timing error factors and equations, develop the oscillator timing model, analyze results, and--if successful--transition this innovation to sponsors/vendors.

Impact
If the model results indicate CSAC/Rb oscillators meet timing requirements, then a prototype could be developed to demonstrate a technical and practical solution for electric grid synchrophasor vendors and other critical infrastructure device vendors (e.g., CDMA/LTE wireless suppliers). The electric grid and other critical infrastructure (e.g., communications) can be protected by mitigating GPS jamming/spoofing threats to prevent wide-scale outages.

Public Release No: 13-0604

Problem
Privacy laws and regulations articulate many privacy requirements at an abstract level, and it can be challenging for system developers to translate these requirements into system characteristics. “Privacy testing" refers to specific system tests that are performed to ensure that privacy requirements are implemented correctly and protect Personally Identifiable Information (PII). Privacy testing is especially vital for systems that process large amounts of Protected Health Information (PHI), and will be increasingly important with the growing use of digital PHI as HITECH Act requirements are implemented. However, there has not yet been a comprehensive effort to articulate privacy requirements at the system/application level and address them using privacy testing to verify that basic privacy controls are correctly implemented within the healthcare environment.

Objectives
"Privacy by Design" is the idea that privacy should be built into systems from the very beginning and considered throughout the system development lifecycle. The goal of this research project is to include privacy requirements definition and testing in standards and guidance documents and to provide a tool for privacy requirements definition and testing efforts that will enable broad adoption within the healthcare industry. This approach will support the adoption of Privacy by Design throughout the healthcare environment.

Activities
We will engage with standards bodies such as the National Institute of Standards & Technology (NIST) and healthcare stakeholders to include healthcare-related privacy requirements and tests in standards and guidance documents used within the healthcare environment. We will revise the existing MITRE privacy risk management tool (PRIME) so that it can be used for privacy requirements definition and testing efforts in the healthcare environment. Then we will engage key sponsors and industry stakeholders in piloting the revised PRIME tool.

Impact
We have identified a general set of privacy requirements and tests and mapped them to the HIPAA Privacy Rule to determine which requirements and tests apply to the healthcare environment and also mapped them to NIST SP 800-53, Appendix J, privacy controls. The team has also begun work on a healthcare-focused demonstration version of PRIME that will be used to obtain feedback from healthcare stakeholders on format and usability. The results of this project will provide healthcare organizations with a tool they can use to include privacy requirements definition and testing in the system development process. This will result in better privacy protection for PHI as organizations will be able to more fully identify and address privacy issues. It will also result in systems development cost savings since errors will be caught earlier in the development process and can be more easily corrected at that time.

Public Release No: 13-0669

Problem
Civilian federal agencies are facing increased pressure to both improve services to citizens and reduce costs. The widespread adoption of Internet and mobile-computing and social media technologies by the public provides new opportunities for improving mission performance and reducing cost, but these technologies simultaneously raise expectations for levels of federal government service. Agency decision makers face the challenge of making critical investment decisions in a highly dynamic and complex environment that includes cumbersome legacy analytic capabilities, systems, and fragmented information. They need a capability that provides insight into how they can change their business processes to take advantage of these technologies while at the same time improving service levels and lowering costs. They also need to access these insights quickly, without expensive and lengthy tests, so that they can implement changes before the technologies become obsolete.

Objectives
We will demonstrate the feasibility and value of modeling and simulation methods to support rapid and effective decision-making on the use of new technologies to dramatically improve citizen-to-government interactions, both in terms of improved mission performance and reduced cost.

Activities
Our research team is working with MITRE subject matter experts (SMEs) in the IRS/Treasury group to identify opportunities to model a system or process with e-Services at the IRS. Once a specific modeling opportunity is identified, the research team will collaborate with subject matter experts on developing influence diagrams to illustrate cascading interactions and feedback loops, which will be used as the design basis for the model. The research team and SMEs will develop scenarios that focus on policy decisions, key feedback loops, operational constraints, and high-level requirements. Then, the team will develop a rapid prototype and run a selected scenario, analyze results, and report findings. The results will be briefed to interested parties at the culmination of the study.

Impact
The research team will develop one or more new or expanded simulation models and examine technical modeling scenarios. We  will demonstrate a quick-turn tool to analyze decision options in a sponsor and customer domain. The model should provide a transparent line-of-sight at the operational and system level.  

Public Release No: 13-0671

Problem
The current SATCOM terminal modem acquisition approach relies heavily on tier 1 DoD contractors who furnish closed architecture proprietary solutions. These solutions often come at a steep price as a result of the ground-up development and customization in both the hardware and software elements. In addition, this closed system proprietary approach prohibits modification, upgrade, or reuse of that system by someone other than the original contractor. This runs contrary to the Modular Open Systems Approach developed by Office of the Secretary of Defense. With the increasing availability of commercial off-the-shelf (COTS) hardware elements, such as Field Programmable Gate Array processing boards, use of appropriate standards, and a suitable reference design, an Open Architecture approach to SATCOM modem acquisition could be realized.

Objectives
Our idea is to move the SATCOM modem vendor and user community toward Open Architecture based hardware development utilizing the OpenVPX standard. This is truly a paradigm shift for this acquisition community. To move forward,  customers at the highest levels must understand the advantages of the approach. At the same time, the potential resistance in the vendor community must be addressed. To support this paradigm shift, we need to obtain a deep understanding of the technology and its application before we can develop a successful acquisition strategy based on an Open Architecture.

Activities
We are focused on the development of a reference prototype based on the Open Architecture reference design that the team developed in FY12. The prototype will feature COTS technology, as well as open standards for firmware and software components. Our goal is to demonstrate the feasibility of the architecture concepts applied to an actual SATCOM modem application. The target application will center on the upcoming Protected Tactical Terminal system for next generation protected SATCOM.

Impact
We have focused on implementation of the data plane functionality using a Serial Rapid IO fabric, which will enable board-to-board connectivity at layers 1 and 2 based on a published open standard. Next we will include the layering of a Digital IF transport layer that will standardize the signal processing data transmission through the modem data plane. In addition, we have implemented a working Simple Network Management Protocol for modem command and control. This will implement a modem management plane in an open standard way that is applicable across a wide variety of communications terminals.

Public Release No: 13-0358

Problem
Traditional systems engineering works very well in many circumstances. However, as the operational environment becomes more complex (i.e., it is made up of many heterogeneous, interacting, potentially adaptive entities), these approaches tend to break down. The Under Secretary of Defense for Acquisition, Technology & Logistics has recognized this and promulgated an initiative called Engineered Resilient Systems (ERS), which are systems that can adapt to many changing environments relatively cheaply and whose performance degrades gracefully. 

Objectives
Motivated by this initiative, we began to look at different ways of conceptualizing an engineered resilient system. For example, an engineered resilient system could be thought of as a species, in which each individual is a solution to one or more particular uses of the system.  If this is the case, we should be able to evolve designs for ERS.  Our Resilient Adaptive Systems (RAS) research team is creating a proof of concept for this idea using biologically inspired heuristic search algorithms, such as genetic algorithms, to evolve a "species" of light infantry vehicles.  However, unlike traditional biological search algorithms in which each member of the "population" is a potential solution, in our work, each solution is a population.  The fitness function will be designed to reward populations that have high performance in a variety of scenarios, few individuals, and low "mutation" cost to transform one individual vehicle configuration into another.

Activities
We have created the attributes of the vehicle, such as armaments, countermeasures, engine size, and transmission type, as well as their potential values.  We are now building up the evaluation function so that each vehicle can be scored on its performance in a variety of scenarios, including: ambushes, checkpoints, convoys, and so on. Furthermore, we are in the process of creating the biologically inspired algorithms that will be used to evolve the species of vehicle.

Impact
If successful, we will demonstrate the viability of this technique to create engineered resilient systems. This ability should, once matured, enable our sponsors to more quickly explore the design space of engineered systems and minimize operations and maintenance costs once fielded.

Public Release No: 13-0707

Problem
Current communication system designs employ advanced signaling techniques to dramatically increase system throughput in a robust and secure manner. In particular, third generation (3G) and fourth generation (4G) cellular standards are deploying multiple antennas at the transmitter and receiver (commonly referred to as MIMO) to push the limits of wireless channels. While these advanced techniques have been shown to dramatically improve throughput and reliability, few analytic results exist to guide the government in incorporating these techniques into existing waveforms, assessing reliability and security in practical scenarios, or developing requirements for new waveforms.

Objectives
There is a need to bridge analytical gaps that exist between relevant MIMO channel models and recent advancements in predictive analytic methods. In particular, no open analytic tool exists sufficient to evaluate the value of incorporating MIMO protocols into government waveforms. Our goal is to unify accurate (analytic) MIMO channel models to predictive tools in information theory, enabling new waveform designs as well as quantitative analysis of existing waveforms.

Activities
Our plan is to do parameter fitting of accepted physical propagation models to more tractable analytical models and conduct physical experimentation with MITRE-owned channel emulators to verify parameter fitting. Then we will develop an analysis tool for practical evaluation of proposed waveforms and refine and/or extend developed models and methods to meet any deficiencies identified through experimentation.

Impact
This work will enable more reliable and efficient contemporary waveform design and evaluation.  In particular, it will provide a toolkit to enable joint MAC/PHY scheduling design for tactical waveforms, the analysis of physical layer security of multiple antenna systems, as well as exemplary MIMO methods of design and analysis under realistic propagation conditions.

Public Release No: 13-0712

Problem
Tactical communication networks are dynamic, infrastructureless, and prone to disruption.  Nevertheless, reliability and end-to-end security mechanisms at the transport and network layers are essential for real-time collaboration and situational awareness applications.  Group-oriented tactical applications (such as chat, situational awareness, and video streaming) are of the highest importance to warfighters, and multicast is the most efficient delivery model for group-oriented communication.  There is no commonly accepted approach or available toolset for securing group-oriented communication in dynamic, infrastructureless tactical communication environments.

Objectives
We will design a flexible architecture for group-oriented tactical applications that does not rely on network infrastructure components.  The architecture will use NSA Suite B cryptography, Datagram TLS (DTLS), IPSec, and the NACK-Oriented Reliable Multicast (NORM) Transport Protocol. We will develop software prototypes of the various architecture components and validate this approach through testing with military realistic group-oriented applications, loading, and mobility scenarios over simulated and real tactical links.

Activities
We plan to fully specify the target multicast architecture and API, and implement prototype software components using available commercial technologies. Then we will demonstrate the security, reliability, and performance improvements achieved through lab-based experimentation

Impact
Availability of an architecture for group-oriented applications that complies with NSA security guidance (i.e., could support communication up to SECRET), makes the most efficient use of bandwidth-constrained tactical links, and can operate in a dynamic, infrastructureless communication environment will have an immediate impact on the design of today's group-oriented applications.  We will work with our DoD mission advocate to transition the multicast architecture

Public Release No: 13-0530

Problem
This team strives to achieve the arbitrary spectral signature modification of a non-planar surface. Traditional approaches for modifying a spectral signature are limited by the electromagnetic properties of materials found in nature or in the properties of materials created by the chemical or mechanical interaction of these materials.  Multilayer-type approaches allow one to create an arbitrary signature but are strongly angular dependent and not readily compatible with non-planar surfaces.   

Objectives
We are pursuing spectral signature modification of non-planar surfaces through the use of engineered electromagnetic materials and structures (EEMS).  Results in the recent literature present experimental demonstrations of EEMS absorbers with high absorptivity over a wide range of incidence angles and independent of polarization.  These properties have been demonstrated at wavelengths ranging from mid-IR to RF. Additionally, these EEMS layers have been fabricated on flexible substrates and recently reported simulations strongly suggest that absorption and transmission properties are maintained when these layers are integrated on curved substrates.  Our desired outcome to advance EEMS materials to the point where it is a practical and implementable solution for the spectral signature modification of non-planar surfaces.

Activities
Planned activities for FY13 include simulations to determine theoretical capabilities and experimental verification of these capabilities.  We will perform simulations to understand the limits of EEMS signature maintenance with respect to substrate curvature. We will also fabricate EEMS absorbers, including the necessary process development and planar spectral testing.

Impact
If successful, this work will allow for the arbitrary signature modification of a variety of surfaces, and thus a variety of objects. 

Public Release No: 13-0382

Problem
It would be useful to understand the dynamics of systems engineering teams and other creative teams to determine what kind of team would fit a situation. While case studies are very useful, creating a large enough sample from which to draw general conclusions is quite difficult, also. 

Objectives
As a possible solution, we are developing a simulation capability that would allow for experimentation with creative problem solving teams. Leveraging the work of Drs. Bednar, Page, and Guimera, we will create an experimental platform that will support the examination of the tensions among team size and coordination, heterogeneity and groupthink, and team composition based on choosing new individuals or previous collaborators. 

Activities
We have implemented the canonical Bednar-Page cultural conformity/consistency model, which examines the tension among individuals wanting to conform with those around them while maintaining a consistent worldview. Our team is also evaluating an implementation of the Guimera model—which examines the tension between building a team with new faces (who have new ideas) and experienced people (who understand the administrative system) to solve problems creatively.  Once that evaluation is complete, we will merge the two models. Then we will test the in silico problem solving teams on a “standard” problem test set.  In this test, the artificial team leads will need to create a team that can accomplish two things to solve the problem: 1) find a solution to the algorithm, 2) understand how to apply the solution “administratively.”  The difficulty of the problem and of navigating the administrative system will be varied and the performance of various team compositions will be measured.

Impact
If successful, this simulation-based experimentation platform will allow for the exploration of the dynamics of creative problem solving in teams that are embedded within an administrative system.  This will help organizations understand, for example, why some teams fail while other do not.

Public Release No: 13-0778

Problem
Recent events around the world have shown that our current communications infrastructure is not as reliable as we would like to believe. Cellular towers can be destroyed by natural phenomena or simply overloaded beyond capacity and Wi-Fi hotspots are reliant on power and network connectivity, two things in short supply during a catastrophic event. We’ve seen these issues surface time and time again, from Katrina to Haiti to Fukushima. It's always the same problem: no connectivity and no communication.

Objectives
Our Smart Phone Ad-Hoc Networking (SPAN) project attempts to ameliorate these issues by providing an alternate means for information dispersal. The project uses Mobile Ad-Hoc Network (MANET) technology to provide a resilient backup framework for communications among individuals when all other infrastructure is unavailable or unreliable. The MANET-based solution is a network with no infrastructure that allows common smart phones to link together in a dynamic way. In this way, the SPAN project is harnessing the ubiquity of smart phones to provide durable communications in times of need. In addition to supporting resilient information sharing, SPAN also allows for “Off Grid” communications--for example,  when transferring and sharing data is necessary but, for security reasons, participants do not wish to use the Internet or existing cellular networks.

Activities
We have created a fully open source framework for implementing and exploring MANET networks, routing algorithms, and mesh-aware applications for Android smart phones. The framework provides a full proof-of-concept implementation of a functional MANET and provides a plug-and-play framework for developing and testing custom ad-hoc routing protocols. The routing protocols are the true cornerstone of the MANET architecture as they adapt the network for scalability, mobility, and power constraints of mobile devices. We are currently working on providing trusted communication paths between remote sponsor entities across an open mesh network to address the need for sharing sensitive information without the administrative and logistical burden of traditional network infrastructure. The team is also working on a secure group voice chat application to replace traditional push-to-talk land mobile radios and iOS support for all of the above.

Impact
We believe we can help solve the problem of limited connectivity and communication during a disaster or emergency. To date we have implemented an open source mobile ad-hoc mesh network for Androids (http://github.com/ProjectSPAN). We have tested our approach at a  hands-on sponsor-facing operation at the 2012  All Star Baseball Game in Kansas City. We have also shared our findings with a wide range of stakeholders--at events such as the 2012 IEEE Conference on Technologies for Homeland Security and the DefCon 20 computer security conference, and with organizations such as the Open Technology Institute at the New America Foundation (Commotion Wireless Project), Flinders University of South Australia Resilient Networks Laboratory (Serval Project); and the Guardian Project, focusing on open source mobile security.

Public Release No: 12-4645

Problem
Strategic planning during times of uncertainty is a key problem for today’s air traffic management system.  Plans are made hours in advance to manage expected weather congestion in order to maintain system safety and throughput.  However, there are often significant uncertainties in both weather and traffic demand predictions at longer planning horizons (4-8 hours in advance).  There are currently no decision-support capabilities to design and evaluate coordinated traffic management plans at these time horizons.

Objectives
In our Flow Contingency Management (FCM) work, we plan to apply a scientific approach to strategic Traffic Flow Management (TFM).  The FCM decision-support system (DSS) enables decision makers to simulate the traffic impact due to weather, as well as the impact of potential mitigation strategies, and to evaluate a strategy’s effectiveness using a variety of metrics.  In addition, FCM will assist the TFM decision-making process by identifying coordinated mitigation strategies that are robust to forecast uncertainties.  

Activities
We are using an FCM DSS prototype to explore methods for providing automation-derived solutions to decision makers so they can focus on the structure of the solution as opposed to the details of the implementation. Research is also continuing on the development of demand and weather-impact forecast models to improve forecast accuracy. We are also exploring visualization capabilities to effectively communicate the problem and proposed solutions.

Impact
The FCM DSS will address deficiencies in the current system by providing an integrated view of the predicted impact of potential future congestion and proposed mitigation plans. Using FCM, decision makers will be able to develop coordinated traffic management plans well in advance of traffic congestion.  Decision makers will be able to communicate the situation and response to National Airspace System stakeholders. Overall, FCM will improve situational awareness for all parties, avoid unnecessary delays, and improve system efficiency.

Public Release No: 13-0426

Problem
The health information technology industry needs a source of realistic health records that do not incur privacy constraints for the development and testing of new systems and capabilities. Although techniques have been developed to de-identify health records, these records still incur legal restrictions, and obfuscation can destroy important temporal or longitudinal data elements. Unrestricted access to realistic healthcare data would catalyze multiple healthcare advances, including 1) standards development and interoperability testing, 2) mobile application development and testing, 3) system scalability testing, 4) verification of quality-of-care metrics including systems designed for measuring meaningful use, 5) privacy and security testing, including emerging areas such as consent-based authorization, and 6) development of health data mining techniques via injection of known or expected patterns into base-case data. The injection approach has been used for detecting adverse events and may be used in the future for detecting prescription drug abuse, disease outbreaks, and billing fraud. The availability of unencumbered medical test data could accelerate many of these areas.

Objectives
Our key objectives are to: -Generate synthetic electronic health records (EHR), free of personally identifiable information (PII) and protected health information (PHI) concerns, suitable to be used for the above purposes; -Make these data publicly available in multiple formats via RESTful APIs by extending the existing hData-based Patient Data Server -Create a model of population health and medical treatment whose fidelity will improve over time via an open source community process -Allow users to create and run their own disease-specific models and simulations to create populations of synthetic individuals suited to particular studies.

Activities
The system under development, called Synthea (for Synthetic Health), uses a novel combination of statistical generation and medical knowledge. Certain elements of the records, such as conditions and medications, can be generated using publicly available probability data, resulting in a partial record containing basic demographic information, conditions, and medications. The partial medical record is then passed through a series of modules that augment and amend the record. The record is enriched by adding encounters, procedures, vital signs and other elements; logical criteria are checked; conflicts are removed; and the record is made more robust and believable. Finally, a record formatter is used to create well-formed EHRs according to pre-specified formatting conventions. The output records are stored in a database, and records can be retrieved, queried, and analyzed by accessing the database through extensions of the existing hData-based Patient Data Server.

Impact
In a nation concerned with the rising cost of healthcare, the increased use of EHRs required by the HITECH Act provides opportunities for process improvements and cost savings in the delivery of health services. In order to facilitate the development and deployment of systems and business processes that make use of and benefit from EHRs, there is a critical need to provide data sets that are free from legal and privacy restrictions that can be used for testing and accrediting new systems and business processes. Synthea will accelerate development of these systems and business processes by providing MITRE sponsors and their stakeholders access to sets of data records that do not incur PII or PHI legal restrictions. The availability of sample medical data in the form of EHRs will help healthcare providers, vendors, and sponsors in a number of diverse areas, including standards compliance, information interoperability and integration testing, clinical system development and testing, security testing, public health data collection, health-related data mining, and medical fraud detection systems testing.

Public Release No: 13-0202

Problem
Within MITRE and other organizations, analytic projects require significant resources to acquire and prepare data for analysis. Furthermore, fine-grained population data can be difficult to obtain due to regulatory or proprietary reasons and the quality of the data sources can be inconsistent.

Objectives
Our team will: -1. Develop a fine-grained synthetic population dataset that is statistically accurate, open, and extensible; -2. Design, develop, and validate an adaptable, extensible, scalable, open architecture to extract statistical profiles from multiple datasets and generate a synthetic population from the statistical profiles; -3. Develop capabilities to facilitate the statistical fusion of disparate datasets. The longer term goal is to host and maintain an open source dataset that is accepted by the community as a standard dataset.

Activities
There are two phases to this work, Phase 1 for FY13, and Phase 2 for FY14. The goal of Phase 1 is to develop a proof of concept to determine the feasibility of developing a synthetic population with an arbitrary number of attributes and of merging separate datasets together. The first step is to develop a consistent mathematical framework for evaluating current algorithms and establish the analytical platform and tools to perform evaluation, design, and development. The second step is to evaluate and extend the current algorithms and identify and acquire two independent datasets for designing and evaluating algorithms. The third step is to prepare the datasets and identify, develop, and test the algorithms for extracting statistical profiles from the data, and for storing and extracting the statistical profiles from the repository. The fourth step is to complete the initial design and the development of the facilities for generating and testing the synthetic population from the stored statistical profiles. In Phase 2, we will extend the work to additional datasets, refine the algorithms for scalability and performance, and design and develop a pre-production prototype.

Impact
A readily available micro-dataset that is consistent, statistically accurate, and devoid of data sharing restrictions will significantly enhance MITRE’s and our sponsors’ ability to evaluate alternative solutions for providing services to the public.  It will also reduce the resources expended each time MITRE performs any analysis that requires population-level data to reach a statistically valid conclusion or recommendation.  This work will also provide a vehicle for the government to share data across agencies by creating a facility for fusing disparate datasets.  In addition, an open standard synthetic population baseline can increase the pool of researchers and analysts working on government-related projects by providing increased availability of data.

Public Release No: 13-0275

Problem
A Traffic Flow Management (TFM) initiative known as an Airspace Flow Program (AFP) reduces airspace demand when severe en route weather is forecast.  The AFP tends to over-restrict the traffic, since predictions of convective weather are unreliable 4-8 hours in advance.  Tactical TFM (TTFM) is a concept for managing air traffic closer in time to the weather.  MITRE has developed a prototype TTFM tool, and there is a need for a benefits assessment.

Objectives
MITRE’s national-level simulation model, systemwideModeler, will be used to model a severe weather day in the National Airspace System to evaluate the benefits of TTFM. The flow rate will be systematically increased  to find the threshold point at which sector workload becomes untenable. We hypothesize that a greater air traffic flow rate will be accommodated under TTFM. Sector workload threaholds will also be defined and validated.

Activities
Planned activities include:  Formulate experiment; work with  Subject Matter Experts to refine the research questions;  determine sector workload thresholds to reflect tactical rerouting and vectoring in weather-impacted sectors; perform production simulation runs; and examine and interpret results.

Impact
If we are successful, our research will help decision makers at the Federal Aviation Administration understand the impact of a tactical rerouting tool and the importance of including such functionality in a future release of the TFM System. 

Public Release No: 13-0210

Problem
Severe weather disrupts normal air traffic flows and causes most air traffic delays. Tactically, air traffic controllers in sectors containing weather implement and manage path changes as traffic nears weather cells. Strategically, traffic management initiatives reduce the quantity of flights traversing impacted airspace by rerouting flights or instituting ground delays to prevent overwhelming workloads in the concentrated weather areas.

Objectives
MITRE is investigating the use of a tactical traffic flow management (TFM) decision support system to reduce weather-induced delays. The concept builds on the accuracy of short-term weather forecasts, which allow for small adjustments in flight routes to avoid storms 30 to 60 minutes before flights reach the weather.  This will move controller workload away from the weather  area and into sectors that have more capacity to implement path changes.

Activities
MITRE will use a tactical TFM prototype to simulate tactical reroutes around all severe weather in the United States for a 24-hour period.  In addition, a national-level simulation model (systemwideModeler) will model all traffic on a severe weather day with and without tactical reroutes.  The modeling adjusts flight flow rates to determine the maximum possible throughout that can be achieved with and without the tactical reroutes.

Impact
The research will show whether the use of tactical TFM reroutes can reduce delays associated with severe weather.  The annual benefit of the delay reduction will be monetized.  The study will determine whether a tactical TFM capability would allow for less restrictive strategic actions (ground delay and large reroutes) and reduce overall system delays.

Public Release No: 13-0738

Problem
There are several challenges in testing the emerging class of networked tactical radios. For example, current approaches leverage simulations that rely on models that often lack implementation-specific behaviors and field tests that lack fine-grain control and repeatability.  To address this, our team is developing lab-based tools that will improve accuracy, which will lead to earlier, more successful deployments of systems of networked radios.

Objectives
If a flexible, extensible, realistic test environment was available, then the system level impact of design and implementation decisions could be more fully evaluated in a controlled, repeatable environment.  We are developing a system that leverages high-speed commercial digital signal processing equipment to enable real-time emulation of complex  radio frequency (RF) environments. We have produced a unique prototype that enables robust testing and evaluation of existing and emerging radio systems.

Activities
We are building on our previously developed prototype to create a system that meets the present and emerging needs of the tactical radio community.  Our prototype is currently installed in an Army radio testing lab and is also in daily use supporting Air Force development efforts. We plan to use their experiences and feedback to focus our development, addressing operationally relevant questions.  We will demonstrate the effectiveness of our approach and create a road map for future development to ensure that both existing and emerging needs are targeted. 

Impact
This work will significantly reduce program cost and schedule by providing access to an advanced radio testbed that enables more thorough lab testing before moving a system to the field.  Our robust infrastructure already provides capabilities that exceed commercial offerings in design flexibility, enabling the reuse of lab assets to supporting different testing needs.

Public Release No: 13-0215

Problem
Massive amounts of data are now available, but existing visualizations do not have the intelligence built in to proactively respond to the current decision-making needs of military units. Per the Assistant Secretary of Defense Research & Engineering Data to Decisions (D2D) S&T Priority Initiatives, users need reactive intelligent interfaces that are adaptable and can draw visual attention to important factors. Additionally, decision makers may not know of significant available information regarding past lessons learned from other units.

Objectives
We are creating an intelligent visualization layer to maximize units’ decision-making effectiveness and assist them in exploiting data currently available at any point in their mission process. This smart presentation layer will recommend a meaningful set of interactive visualizations and associated data transformations and will generate new recommendations as the user progresses through the process. The recommended visualizations will help draw users’ attention to the aspects most critical for the decision at hand.

Activities
We will continue building our prototype system (Tailor), which will consist of: a widget that provides the user control over whether to accept recommendations for adapting their current user interface, a RESTful web service that gets notified of the user’s current step in the process and returns a set of recommended visualization widgets and data transformations to support the current decision, and a set of option awareness visualizations to support selecting a course of action.

Impact
DoD sponsors will benefit from a system that reacts and adapts visualizations to take advantage of the increasing amount of data available and draw the user’s attention to the most critical aspects. Our prototype system is intended to assist military units by providing what computer scientist Ben Shneiderman describes: “The eyes have it: although the computer contributes to the information explosion, it is potentially the magic lens for finding, sorting, filtering, and presenting the relevant items.” 

Public Release No: 12-4908

Problem
Today, in order to predict outcomes of taxpayer outreach or compliance approaches, the IRS is limited to surveys and focus groups and making assumptions based on previous tax filings.

Objectives
This approach to taxpayer behavior modeling and simulation using virtual reality based serious games provides the IRS with a new kind of tool for determining potential success of their approaches before risking large implementation costs.

Activities
To fully demonstrate and transition this approach to the IRS we will conduct a full experiment - in collaboration with George Mason University - with at least 30 student test subjects playing alternative (control and test) versions of our virtual reality based business simulation game to test a sample hypothesis that has been reviewed and approved by the IRS.

Impact
The Tax Ecosystem Modeling Using Virtual Reality Environments MIP project has now successfully demonstrated that serious game techniques, using the virtual reality Open Simulator environment, can simulate small business taxpayer behavior to allow the IRS to collect behavioral data to test hypotheses around alternative approaches to taxpayer outreach and compliance. The business simulation game that allows this has been implemented and beta tested, and we have already demonstrated it for high level IRS executives who have shown great interest in using it to test hypotheses of interest to them. A serious game virtual reality based simulation approach will provide the IRS with behavioral data that will allow them to test - beyond mere surveys and focus groups - the affect on taxpayer behavior of potential taxpayer outreach and compliance approaches. In addition, a virtual environment will allow for collaborative exploration and visualization of the results, because the IRS can observe taxpayer behavior from inside the game and the virtual environment.

Public Release No: 13-0626

Problem
Trajectory Based Operations (TBO) is the NextGen concept of improving throughput, flight efficiency, flight times, and schedule predictability through better prediction and coordination of aircraft trajectories.  Analyzing the integrated benefits of TBO has proven to be a serious challenge. It has been difficult for analysts and decision-makers to visualize system impacts and relationships among capabilities, and differences in intuition on tradeoffs has made consensus building difficult.

Objectives
We are developing a mathematical framework for TBO in which benefits, constraints, and tradeoffs are described by the magnitude and timing of aircraft trajectory uncertainties and by the impact on air traffic management performance metrics from uncertainties at different look-ahead times. This framework will enable uncertainties and impacts from today’s National Airspace System (NAS) to be considered in characterizing proposed TBO capabilities, and will enable individual and cross-capability performance tradeoffs to be quantified and more broadly understood.

Activities
We are developing methods to interpret historical data on aircraft flights to measure what predictive uncertainty was at the time for different look-ahead horizons, flights, locations, and NAS-actor perspectives. We will validate results by comparing with error statistics for arrival time predictions downlinked by aircraft automation. Moving forward, we will focus on completing demonstrations of the framework and positioning it to support appropriate MITRE work.

Impact
The analytical framework for TBO will enable: individual NextGen capabilities to optimize timing performance tradeoffs, portfolio-level trade-offs among alternative trajectory management capabilities, and system-of-systems timing performance optimization for NextGen.

Public Release No: 13-0762

Problem
Obtaining common situational awareness has been repeatedly identified as an operational shortfall by Federal Aviation Administration Traffic Managers (TMs). Based on multiple sources of information and experience, TMs develop and maintain a mental picture of the operation and identify impending problems and potential solutions. To coordinate a solution, they verbally communicate their “picture” to others who are impacted and who have their own views and experience. Differences between these views must be reconciled to reach a decision.

Objectives
This research effort will investigate methods for providing common performance-based situational awareness of traffic flow management initiatives to decision makers. The vision is to provide a quick reference dashboard of relevant operational information that enables fusing these current actions into a comprehensive, shareable, integrated system view.

Activities
Our research team is developing the concept of use, performing data exchange analysis to determine required interfaces and data elements, and developing algorithms for performance metrics generation and presentation. We will use table top exercises to identify desired status information, performance metrics, and data integration and display initiatives. Prototype development and mini-evaluations will be used to gain perspective on the operational feasibility and procedural and policy impacts of the concept.

Impact
Successful development and operational acceptance of this dashboard is expected to yield many qualitative benefits, including providing a shared system view and objective (system generated) analysis of effectiveness of initiatives in place allowing for a common frame of reference for all affected personnel/facilities.  

Public Release No: 13-0456

Problem
With the application of hydraulic fracturing and horizontal drilling, new natural gas shale formations are becoming widely available in North America at competitive prices. This development has created an extensive search for new markets in which natural gas may compete effectively with other energy sources. Recent government forecasts suggest that this transformation may have mixed effects on carbon dioxide emissions. Although natural gas is less carbon intensive than other fossil fuels, it is more carbon intensive than nuclear, renewables, and energy efficiency.   Key drivers on this topic include the following. 1) The future availability of natural gas resources depends on technical progress in discovery and extraction and drilling restrictions due to drinking water safety, water availability and greenhouse gas emissions.  2) The conditions and policies that influence fuel use decisions. 3) Demand for natural gas will be larger with faster retirements of coal power plants, reduced siting of renewable power facilities, more stringent regulation of mercury, particulates and other emissions, and maintained current price differentials between oil and natural gas. The reverse conditions would weaken future natural gas demand growth.

Objectives
MITRE is collaborating with Stanford University’s Energy Modeling Forum, including 22 energy modeling teams from across the globe. This project will examine research questions such as: -What are the policy and technical factors that affect the competition in liquid fuel markets (biofuels, gas-to-liquids, oil shale) and what are the potential impacts on world oil prices? -What incentives are driving agriculturists, biofuel producers, gas production, oil refiners and the U.S. consumers? -What does it mean for other liquid fuels for transportation: Gas-to-liquids, natural gas, biofuels (cellulosic), shale-oil trade-offs? -What infrastructure requirements are necessary to support emerging diverse liquid fuel supplies, markets, and end use (refineries, pipelines, roads/rail)? -What is the impact of ethanol expansion in the United States on agricultural commodity prices? What's the impact on U.S. ethanol industry and jobs (by region) and derivatives? -What are the impacts of changing our trade policies on ethanol imports (tariffs/subsidies)? -Is long-term domestic production of ethanol competitive?  What are the price-points/thresholds as compared with other feed stocks? -What are the alternatives to volume-based ethanol blending? -What is the impact on the environmental footprint of ethanol?

Activities
This work is a cooperative research project that will be developed with a selected government sponsor and validated by subject matter experts. The research plan will follow the steps outlined below: -Identify a government sponsor and agree on the service systems to be studied; -Meet with subject matter experts to define the proposed service description(s) and identify data requirements; -Survey applicable policies and legislation and add it to the MITRE Energy Tax/Subsidy Model; -Develop a conceptual model that describes the research scope and approach; -Iterate through several rapid prototype development cycles to verify and validate the model with subject matter experts; -Identify the simulation experiments with a final set of assumptions and data; -Run the experiments and share the results broadly.

Impact
These results could provide insights into how to achieve the goals of fuel diversification and domestic production while maximizing the positive economic impacts and reducing the unintended consequences of the program.

Public Release No: 13-0679

Problem
The TooCAAn (Toolkit for Corrective Active Annotation) project represents a multiyear effort to transform the way gold-standard annotated text corpora are created by providing powerful tools that are accessible and appealing to non-expert users.

Objectives
In FY12, we expanded our toolset to support complex annotation tasks involving annotation of relationships, such as co-reference between mentions, biographical relationships, or events. We have found that these relation-style tasks are almost always composites of multiple smaller tasks, sometimes with multiple reasonable completion orders. For such tasks, the challenge of coordinating document flow while maintaining the quality and consistency of annotations adds another layer of complexity to the task of supporting annotation, and the challenge is even greater when the workflows involve varied combinations of fully automated, fully manual, and mixed-initiative steps.

Activities
In FY13, we are extending our tools to allow users to express dependencies between sub-tasks and to either enforce a particular step order or allow any step order that satisfies the specified dependencies.  We have developed a declarative representation for describing annotation sets in XML-based annotation set descriptors, which can be composed together into a complex task, aligning each descriptor with one or more steps in the workflow. 

Impact
In our FY13 work, we will extend the existing task model to support this alignment and update our existing tools to better support a step-by-step approach.  This will ensure more correct and efficient annotation and will minimize error propagation by enabling and encouraging error correction at each step of the process.

Public Release No: 13-0148

Problem
Electromagnetic bandgap (EBG) surfaces have found widespread use as alternative ground planes in antenna applications and are particularly useful for low-profile and conformal antennas.  Usually, the concept behind such designs is that the EBG surface acts as an artificial magnetic conductor in the operational bandwidth, which means that it appears to have high surface impedance (typical metal ground planes have very low surface impedance). EBG ground planes’ high apparent surface impedance enables placement of canonical planar antennas with two-sided patterns near a ground plane for a variety of new low-profile, wideband applications.  However, it remains difficult to predict the behavior of EBG surfaces in an antenna environment, because characterization methods largely depend on far-field properties of the surface (most notably reflection phase) while the antenna interacts with the surface in the near field.  The typical result is that the antenna’s expected behavior is detuned and the antenna must be modified to work with the EBG surface.

Objectives
Our goal is to develop a method of designing EBG-backed antennas that natively incorporates the near-field interaction between the antenna and EBG surface. The result will be a design technique that is based on an understanding of the guided waves on the combined antenna-EBG structure, effectively designing the antenna and EBG surface in tandem rather than modifying the antenna to fit the EBG surface.

Activities
We began by developing a generalized guided-wave analysis of a periodically loaded microstrip line that models the behavior of a microstrip line above an EBG surface. We will then use electromagnetic simulation to benchmark the model against published EBG-backed antenna designs to identify the relationship between antenna behavior and regions of the EBG-backed line’s band structure.  Based on those results, we will develop a design technique that considers and incorporates the EBG surface’s modes.  Finally, the design technique will be used to design and fabricate a proof-of-concept antenna.

Impact
Analysis of the guided wave behavior and bandwidth properties of EBG structures will help resolve an open question in the applied electromagnetics community.  Presently, there is a gap in understanding between characterization of EBG structures (through their far-field behavior) and integration of those structures in antenna designs.  Our work will facilitate the design of new low-profile and conformal antenna types.

Public Release No: 13-0065

Problem
Prescription order-entry systems use both structured and free-text fields. This offers physicians necessary flexibility but risks discrepancies between structured fields and natural language text. Researchers at the Veterans Health Administration  and Partners Healthcare have found that these discrepancies occur in 5 to 16 percent of prescriptions. These errors affect costs, time, and patient health; for example, 1.5 million preventable adverse drug events occur annually (costing $3.5 billion), more than half of which are due to process errors during order entry.

Objectives
An automated detection system at the point of entry could increase patient safety and reduce the burden on both pharmacists and physicians. Statistical and rule-based artificial intelligence, accessed via a web API, could provide monitoring-as-a-service, functioning as a clinical alert backend for order-entry applications. Prescription data is characterized by being semi-structured (there are both free-text and structured data fields) and semi-redundant (information may or may not be repeated between field types). The TranScript project team is developing and evaluating techniques for detecting inconsistencies under such conditions, providing the validation component for this alert system.

Activities
We are integrating the automated inconsistency detection system with a computerized order-entry system through a pilot with a clinical partner, and we are researching unsupervised methods for training discrepancy-finding models on data without gold standard labels. We plan to transition prototypes and release code to sponsors and the open source community. We will also explore applications to other semi-structured, semi-redundant data types in the healthcare domain, including fraud detection based on discrepancies between medical narratives and billing codes.

Impact
The TranScript prototype is capable of detecting discrepancies in gold-standard data with non-trivial precision (measured against gold-standard data), demonstrating that artificial intelligence approaches can improve the robustness of the prescribing process. Automated discrepancy detection will save money, time, and, most importantly, patients' lives, benefitting care providers (including the Veterans Health Administration), payers (including Medicare and Medicaid), and U.S. residents.

Public Release No: 13-0265

Problem
Structured Analytic Techniques (SATs) are designed to help intelligence analysts mitigate cognitive limitations and biases that inhibit effective analysis. The intelligence community (IC) widely promotes the use of SATs and provides training for intelligence analysts in a variety of SATs. Yet the IC has not pursued systematic empirical evaluation of whether or not these techniques are useful.

Objectives
We are creating and evaluating an approach to empirically test the effectiveness of SATs, which the IC can execute with little or no outside support. A key goal is to minimize inevitable tradeoffs between internal validity (extent to which cause and effect findings are warranted) and external validity (extent to which findings can be generalized) that have hindered acceptance of previous research on analytic method effectiveness. We are also devising a scientifically sound approach to compare experimental outcomes to “ground truth” answers.

Activities
In collaboration with intelligence analysts, tradecraft experts, analytic method trainers, and social scientists, we are developing an experimental approach that can be transitioned to the IC. During the first year we will document challenges to testing SATs in the IC, articulate an approach to address these challenges, and pilot test an experimental design to inform testing during the second year.

Impact
If the approach proves to be useful, the IC will have a means to gather evidence on the extent to which SATs improve analysis and under what conditions. The research could broadly impact not only how the IC tests analytic methods, but their selection, teaching, and use.

Public Release No: 13-0309

Problem
While information quality metrics and techniques help CIOs and other senior IT managers assess and improve the quality of enterprise data over months or years, in most of this work there is an underlying assumption that time is not a critical aspect of information quality. Our research considers the information quality needs of a different type of customer: the time-constrained builder who does not control any information assets of his own but who instead must quickly choose the most promising sources of information to harness, combine them, and present the information to users.  We propose a new approach to the “quality” of a source.  The metric we want is “time till the source is online, providing value to the organization.”  Variants might include “time till 90 percent of the source’s data is online, providing value to the organization,” or “fraction of the source’s data than can be made available in various times.” Alternatively, we might ask: "given a finite amount of skilled labor (e.g., 2 staff days), what would be the highest value data cleaning activities to pursue to make the resulting data as useful as possible?"

Objectives
If we can create a set of metrics that can quickly and easily categorize ways in which the data is dirty, and how dirty it is, operational costs and disasters can be kept to a minimum by allowing for better integration, usage choices, and planning. Knowing how dirty a data resource is, and in what particular aspects it is dirty, allows users to make informed decisions about its use (or choose an alternate), to manage development resources better, and to create strategies to minimize the impact of dirty data on an application. Not all data quality problems can be easily fixed, but at a minimum developers and users can be aware of what they are, and what that means about the usability of their results.  In this project we will: design a set of metrics that encapsulate several different measures of data dirtiness; create a tool that allows end users, system evaluators, and developers to quickly and easily understand the strengths and weaknesses of a dataset; and investigate potential automatic cleaning and mitigation strategies.

Activities
First, we will build a framework through which a user or application can specify the datasets and constraints of interest. For instance, the user may specify that the dataset must contain good quality road information and be fast to integrate. Using this infrastructure, we will map the constraints to a set of classic data quality and information profiling tests to provide an answer to the specific constraints – both in terms of classic data quality and integration readiness. Finally, we will provide the ability within the tool to schedule the best tests to run in the time allotted -- thus if there is time to provide a more accurate reading, we will run the better test.

Impact
When someone in the field needs to rapidly compose a new capability with novel data sources, this tool will provide the ability to evaluate the strengths and mitigate the weaknesses of the data they wish to use immediately. The results of these tests will be presented to the user, so that the user can make an informed choice.  For instance, the engineer trying to help build a capability to locate evacuees may choose to ignore some data issues if the schema is particularly sound, since it makes the coupling and integration step easier and faster.  Tradeoffs will likely always be needed in the name of expediency, and arming users with this information, they can choose the best tradeoff for their situation. This will lead to better management of electronic resources and better operational outcomes.

Public Release No: 13-0071

Problem
Business General Aviation (GA) traffic comprises a large percentage of air traffic in the National Airspace System (NAS). Because of the sensitivity and widely distributed nature of these business operations, there is little capability to systematically apply insight about the drivers and value proposition from the Business GA operator perspective when planning for future NAS enhancements. Without understanding their perspectives, many opportunities to achieve future NAS benefits, or to accelerate future NAS benefits, may be lost.

Objectives
The hypothesis of this research is that Business GA operators place high value on access and safety. We will develop case studies that will allow us to develop value curves representing the Business GA response to changes in access, predictability, and time and operation cost savings to be generated. We also plan to develop a framework to address specific questions on how Business GA itineraries impact the NAS today and in the future.

Activities
We will identify scenarios in which Business GA is a significant factor in local or system-wide performance. These scenarios will be used to conduct a value chain analysis, and the results will be used to derive metrics that can help predict Business GA responses to NextGen system improvements, beyond the case studies. We will also generate Business GA trajectories to represent synthetic future itineraries that capture the operational characteristics of Business GA.

Impact
The products of this research will include improved modeling of Business GA trajectories and itineraries, modeling of Business GA responses to aviation policy, and expanded NAS metrics. Increasing knowledge in this area will allow MITRE and the FAA to incorporate Business GA perspectives into policy decisions. It may also result in opportunities to accelerate future technology equipage and to propose novel approaches to leveraging Business GA operators as partners in NAS modernization.

Public Release No: 13-0700

Problem
There is a rich body of knowledge locked in patient records and in the biomedical literature and biomedical databases. Because much of this information is in free text, it is currently inaccessible for large-scale analytics. The development of cost-effective information extraction and curation of this information will make it possible to identify and interpret relations between a patient’s genotype (genetic make-up) and phenotype (physical characteristics, including patient health status). This is key to research in personalized medicine, understanding disease processes, identification of effective therapeutics, and selection of patient cohorts for clinical trials.

Objectives
We are using a novel hybrid approach to curation that combines automated biological entity extraction (e.g., genes, mutations, diseases) with crowdsourced curation of relations among these entities to capture phenotype (disease)/genotype relations. Our initial experiments indicate that crowdsourcing can be an effective way to rapidly curate relations among these entities, even for judgments requiring biomedical domain.

Activities
In our second experiment on gene-mutation curation, we used an automated gene extraction module from the National Center for Biotechnology Informatics and automated mutation extraction (EMU from the University of Maryland, Baltimore County), followed by crowdsourcing using “pre-qualified” annotators, whom we recruited via the Amazon Mechanical Turk. Results showed over 85 percent accuracy at a cost of ~50¢ per Turker per abstract.  Our next experiments will focus on scalable, cost-effective curation for gene-mutation-disease relations. 

Impact
Scalable cost-effective biomedical curation can enable capture of patient phenotype/genotype relations. This information is critical for the development of new diagnostics and novel therapeutics tailored to specific patient genotypes, leading to improved outcomes. Our work supports development of an open repository of genetic variants and associated phenotypes, which is critical to translational research such as the NIH-funded eMERGE consortium (electronic Medical Records and Genomics), the Pharmacogenomics Research Network, or the VA’s Million Veteran Program.

Public Release No: 13-0334

Problem
Making decisions within complex problem domains requires effective collaboration across multiple stakeholders to understand the problem space and identify and experiment with potential solutions. This exploration and decision process entails multiple cognitive, evaluative, and affective activities that are often parallel and interactive.  To transform their business operations, government agencies need a fast, inexpensive, and systematic way to explore their complex problem domains and rapidly develop models that allow them to experiment with different solution options and to analyze the results that lead to data-driven decisions. 

Objectives
In FY10, MITRE began to develop a capability called the Virtual Business Experimentation Environment (VBEE) to enable our sponsors to apply innovation and experimentation through a repeatable and sustainable methodology to identify and evaluate opportunities for transforming their business operations.  In FY13, the team is refining the VBEE methodology to emphasize effective techniques for collaborative problem exploration and rapid development of models for experimentation.  We will modernize the suite of collaborative tools to support the problem exploration, solution identification and evaluation, and analysis phases of the VBEE methodology.  We are also documenting the use of the VBEE methodology and toolset so that we can transition this capability to the government, and use it across MITRE.

Activities
Our work will include: Enhanced Collaboration Tools:  continue to work with the Continuous Immersive Systems Engineering research team to develop and beta-test a mobile application that allows for POET evaluations throughout the VBEE engagement process, and to develop an integrated suite of virtual, mobile tools to enable collaborative problem exploration and analysis. -Mission Information Repository: design and develop an implementation plan for a persistent capability that meets both internal and sponsor requirements for storing, cataloging, and sharing artifacts, including models and their data sets, from all VBEE engagements. -Problem Exploration and Evidenced-Based Decision Research:  refine the problem exploration phase of the methodology to integrate findings from related research, including research results on analysis-driven innovation and decision-making approaches. -Transition to Sponsor Environment: document the VBEE methodology and use of tool sets for reuse by our sponsors and within MITRE. 

Impact
Our sponsors will have a proven methodology, supported by relevant tools and technologies, to rapidly examine business problems and develop and evaluate solution alternatives to make data-driven decisions.  This methodology and toolset will support collaborative problem exploration, modeling, and analysis for government organizations and industry.

Public Release No: 13-0288

Problem
In a number of settings, the problem of intelligence extraction is becoming more difficult because the target source is using more sophisticated systems with adaptive algorithms. These systems have the flexibility to reconfigure their internal structure and parameters to satisfy the mission objectives. For example, software defined radio is a flexible system that can adjust the radio waveform based on the surrounding environmental conditions. If we do not re-evaluate our current technology with respect to this flexibility, then we risk our intelligence gathering approaches becoming obsolete.

Objectives
To extract intelligence from flexible and adaptive target sources, we propose to develop a general automated exploitation framework that is flexible and easily adaptable. Specifically, we propose to research a probabilistic framework using Bayesian inference and machine learning techniques. This goal necessitates an open architecture to utilize both legacy and future algorithms. An open architecture gives us the flexibility and adaptability that we need to contend with these versatile target sources.

Activities
Our plan is to: -Establish an abstract, generalized framework -Create methodology to identify “good” inference algorithms to use with the framework -Develop tools to analyze and predict/bound the performance of the framework.

Impact
Our work will help ensure that intelligence gathering and exploitation remain relevant in the presence of flexible and adaptive target sources. In addition, it will provide a solution to reduce the workload of the human analyst.

Public Release No: 13-0400