Human Language -- Projects

Answer Extraction for Information Retrieval

Babylon

Communicator

Conceptual Browsing

Contact Center of the Future Laboratory and Demonstration

Reading Comprehension: Reading, Learning, Teaching

Reference Resolution in Multimodal User Interface to Map-Based Applications

Speech in Noisy Environments (SPINE)

TIDES (Translingual Information Detection, Extraction and Summarization)

Understanding Words You've Never Heard

Human Language researches computer systems that understand and/or synthesize spoken and written human languages. Included in this area are speech processing (recognition, understanding, and synthesis), information extraction, handwriting recognition, machine translation, text summarization, and language generation.

Answer Extraction for Information Retrieval

Randy Fish, Principal Investigator

Bedford and Washington

Problem
We have developed the QANDA question answering system and have evaluated it in government Q&A competitions. What remains to be done is to determine how the QANDA system performs in real-world applications. To date no one working on Q&A has put their system in the hands of real users with the goal of determining real-world performance.

Objectives
We will develop example scenarios with our target users and collect performance metrics while test subjects perform the duties specified in these scenarios using both QANDA and a control system such as Key Word Search technology. We will assess parameters such as the quality of the user's performance of the task, the total elapsed time required, the number of questions asked, etc.

Activities
Project activities include defining the evaluation process, adding instrumentation to the system to record evaluation parameters, porting QANDA to two target sponsor environments, conducting internal trials, conducting sponsor trials, and evaluating the results.

Impacts
The knowledge gained in this final phase of the QANDA project will allow us to advise our sponsors intelligently on when, where, and how QANDA or a COTS Q&A system can impact their work. This will also allow us to prioritize our future research agenda in question answering by assessing which initiatives are most likely to translate into benefits for the end user.

Babylon

Joyce D. Williams, Principal Investigator

Washington only

Problem
Oral communication between people of different linguistic backgrounds is essential in many areas of modern life, including in many situations involving US government personnel in foreign countries. Since all people prefer to use their native language to communicate, it is essential to provide the technical capability to allow all participants in a conversation to speak their native languages.

Objectives
The primary objective is to develop hand-held speech-to-speech translation systems that can be immediately deployed for use in tactical environments in which English speakers need to communicate with speakers of other languages. Babylon will focus on creating the systems in six to eight target languages (Dari, Farsi, Arabic, Mandarin, Pashto, among others) in three tactical domains (force protection, medical triage, and refugee processing).

Activities
MITRE supports the development of the evaluation protocols for the Babylon systems to assist DARPA in determining the efficacy of the various software systems developed by four Babylon contractors. In addition, MITRE will serve as a liaison to subject matter experts in each of the three domains of interest to assist DARPA in determining the needs of the tactical users.

Impacts
As advisors to the Babylon data collection and evaluation process, MITRE brings its extensive expertise to bear on an area of critical national need.

Communicator

Lynette Hirschman, Principal Investigator

Bedford and Washington

Problem
In a world with more and more information at our fingertips, natural, flexible access via widely available, mobile low-cost devices (such as the telephone) is crucial to providing the most effective distribution of information access at the lowest cost.

Objectives
The mission of the DARPA Communicator program is to extend the state of the art in spoken, mixed-initiative dialogue systems. Success will mean faster and easier methods for information access by military operators. The specific application for Communicator is to enable people to converse with computers in order to create, access, and manage information and to solve problems.

Activities
MITRE's contribution to the DARPA Communicator program includes regular delivery and maintenance of the Galaxy Communicator Software Infrastructure (GCSI), which underlies the dialogue systems built by Communicator participants, as well as related tools. MITRE's contribution also includes publishable research on the contrasts between human-human and human-computer dialogue.

Impacts
MITRE's support of the GCSI helps to build a lasting community of researchers, developers and engineers in mixed-initiative spoken dialogue systems, helps to "lower the bar to entry" for new researchers, and facilitates the emergence of standards and best practice in dialogue system design and construction. MITRE's dialogue research helps to identify requirements for current and future dialogue-oriented tasks.

Conceptual Browsing

Inderjeet Mani, Principal Investigator

Bedford and Washington

Problem
Many of our customers are embracing e-commerce applications that depend on effective characterization of information content and efficient access to relevant information. In current Web-based information-access systems, such as Yahoo, the cataloging systems depend upon manual organization and structuring of information into ontologies. However, manual organization is expensive.

Objectives
Our goal is to automatically induce ontologies for information access. We will use a combination of statistical and linguistic methods to identify important terms in document collections and to discover inter-term relationships. We will evaluate the resulting ontologies by asking human subjects to judge relationships inferred by the system, as well as by automatically comparing machine-generated and human-generated ontologies.

Activities
The project has evaluated a domain-independent term discovery method, developed initial knowledge sources for discovering relationships, and integrated them with Veridian's ThemeLink search engine. We also enhanced the algorithms and applied them to a variety of domains. We have completed an initial set of evaluation experiments. We will continue with further evaluation experiments, apply our algorithms to other domains such as bioinformatics and book index generation, and report on transition efforts.

Impacts
This technology can provide for improved browsing capabilities for text collections. It can also enable a variety of Web-enabled e-commerce applications to provide a higher quality of cataloging through the automatic organization of information. This research may also be applicable to the QANDA project. We expect to transition these technologies to the IRS.

Contact Center of the Future Laboratory and Demonstration

David L. Madison, Principal Investigator

Washington only

Problem
The IRS is under congressional mandate to improve the quality of its customer communications services, but must do so under severe budget constraints. Current telephony call centers must be modernized to provide accurate, consistent responses to multilingual queries coming in through multiple communication channels. A high degree of automation will be required to contain operational costs.

Objectives
The objective of this project is to build and demonstrate a Contact Center of the Future (CCOF) Laboratory that incorporates emerging contact-center technologies such as VoIP and Voice XML. The laboratory and demonstration will help to establish a partnership between MITRE and the IRS to develop a vision of modernized customer communication services.

Activities
Three activity tracks have been launched to accomplish the CCOF objectives: 1) design and implement a CCOF Laboratory consisting of telephony, networking, and commercial off-the-shelf (COTS) software components; 2) develop IRS-specific call-routing and question-answering advanced applications; and 3) integrate the advanced applications with the CCOF Laboratory for testing and demonstration purposes.

Impacts
The CCOF Laboratory and demonstration will provide an entrée for future MITRE work with the IRS in establishing a clear vision of the next generation of customer communication services. The demonstration will help to establish MITRE's credentials in contact/call center technology, and the CCOF Laboratory will allow testing of new contact-center concepts and products.

Reading Comprehension: Reading, Learning, Teaching

Lynette Hirschman, Principal Investigator

Bedford and Washington

Problem
This project is addressing a three-stage grand challenge application for human language technology: building a system that can "learn to read," then "read to learn" and finally "teach to learn." This project addresses issues of machine learning, knowledge acquisition and instructional technology.

Objectives
Our first objective is to build a computer-based system that is capable of passing a third grade reading-comprehension test. Second we will build a system that will "read to learn," passing a test on that subject matter after having read the text. Finally we will build a system that can learn through interacting with a person, and at the same time, help to teach the person.

Activities
We have applied prototype systems on reading comprehension tests designed for fourth to eighth graders with a 30%-40% accuracy. We are improving the system to include more components. We will implement a reciprocal teaching demonstration, where the system plays the role of teacher (grading student answers) or the role of peer learner (answering questions posed by a real student).

Impacts
This research will open new areas of research, addressing issues of machine learning, breaking the knowledge acquisition bottleneck, developing new evaluation measures for understanding and learning, and creating new instructional technologies via learning companions and interactive teaching environments.

Reference Resolution in Multimodal User Interface to Map-Based Applications

Lisa Harper, Principal Investigator

Washington only

Problem
Advanced human-computer interface systems that interpret graphical, gestural, and language-based input are of growing interest to MITRE sponsors. One problem faced by these systems is resolving references across modalities. For example, users may refer to an element on a map using natural language or by pointing and clicking. A problem for multimodal understanding is that information across modalities must be fused.

Objectives
The purpose of this project is to investigate and develop a framework for the interpretation of graphical, gestural, and language-based input for semantic understanding. The research domain will be map-based interaction.

Activities
Our research plan consists of two major thrusts: (1) a corpus study of multimodal interaction and (2) a prototype research system utilizing gesture and speech input for conversing with an artificial agent in map-based interaction. In our third year, we will develop an integrated discourse understanding module demonstrating multimodal fusion in a robot control task.

Impacts
MITRE is a leader in natural language and multimedia processing. We will leverage these strengths to create a leadership role in the new field of multimodal understanding. The potential applicability for our sponsors is great. Areas that multimodal reference resolution would benefit include such diverse domains as simulations, command/control, geospatial systems, computer-based learning and text analysis.

Speech in Noisy Environments (SPINE)

Bryan George, Principal Investigator

Washington only

Problem
Voice-driven tactical human computer interface and intelligent information access applications are characterized by harsh environmental conditions such as high background noise, vibration, stress, task loading and interference. Robust, reliable, automatic speech recognition (ASR) technology is critical to the successful deployment of voice-driven systems for military use.

Objectives
The primary objective of the DARPA SPINE program is to improve the utility of ASR in multi-speaker, high noise environments encountered in tactical applications. To achieve its primary objective, the SPINE program will seek fundamental advances in ASR technology areas including sensor/transducer systems, speech signal processing, robust feature extraction, and robust acoustic, language, and discourse modeling.

Activities
MITRE will develop corpora to promote fundamental scientific and engineering advances in robust ASR for tactical applications, conduct formal evaluation of SPINE R&D results and technologies, and develop a flexible, open software infrastructure for robust ASR to assist SPINE researchers and to promote wider and more consistent use of robust ASR technology in a broad range of computing applications.

Impacts
MITRE's role in data development and evaluations will help define the course of robust ASR research in the SPINE program and beyond. We expect the open ASR infrastructure to promote collaborative research and development in robust ASR, and to promote the emergence of best practices and standards in robust ASR system design and implementation.

TIDES (Translingual Information Detection Extraction Summarization)

Lynette Hirschman, Principal Investigator

Bedford and Washington

Problem
Over the years, expanded trade and travel have increased the potential economic and political impacts of major disease outbreaks. More recently, the potential of biological terrorism has become a very real threat. Appropriate response to disease outbreaks and emerging threats depends on obtaining reliable and up-to-date information, which often means monitoring many news sources, particularly local news sources, in many languages worldwide.

Objectives
The goal of the TIDES program is to provide information on demand, independent of language or medium. Under TIDES, MITRE has developed the MITAP (MITRE Text and Audio Processing) system, which captures and processes global on-line information (including news and email) to provide situation awareness for monitoring biological and other threats.

Activities
MITAP captures over 90 sources (newswire, email, broadcast news) in eight languages and automatically filters, translates, summarizes, and categorizes messages into searchable newsgroups based on disease, region, information source, person, and organization. Critical information is automatically extracted and tagged to facilitate browsing and sorting. A general search engine supports key word search and ordering of results by date or relevance.

Impacts
MITAP is an operational prototype created for tracking infectious disease outbreaks and other global threats. MITAP focuses on providing timely multi-lingual, global information access to analysts, medical experts, government users, and humanitarian organizations. A MITAP product, the World Press Update, is distributed to over 120 readers, including decision-makers in the government and military.

Understanding Words You've Never Heard

John C. Henderson, Principal Investigator

Bedford and Washington

Problem
Current state-of-the-art techniques in automatic speech recognition (ASR) explicitly model all words in a pre-defined vocabulary list. The ASR system processes the acoustic speech waveform and identifies the vocabulary word that most closely matches the acoustics. Out-of-vocabulary (OOV) words, spoken words that are not explicitly modeled in the ASR vocabulary, are guaranteed to result in output errors.

Objectives
In this project we are developing new techniques for addressing the OOV problem in speech recognition. Our research focuses on identifying post-processing methods for OOV error correction that do not require direct modification of the ASR system itself. We are developing probabilistic models of natural language information content, ASR error patterns, and the phonetic characteristics of speech recognition errors.

Activities
Our research addresses three major tasks: information extraction, error detection, and error correction. Information extraction involves identifying regions of the ASR output that are likely to contain significant information, such as proper names. Error detection involves identifying which words within the name phrases are ASR errors. Error correction involves using phonetic distance calculations to correct the ASR errors within name phrases.

Impacts
Currently there is no technique available for addressing the OOV word problem in ASR systems other than to add OOV words to the system vocabulary. However, adding words to the vocabulary can actually hurt system performance. Our approach is the first research focusing on post-processing of ASR errors, a technique that can be applied to any ASR system with any vocabulary size.