Human Language -- Projects

Babylon

Core Dialogue Research

Foreign Language Tool Improvement Through Evaluation - FLITE

Reading Comprehension: Reading, Learning, Teaching

TIA (Total Information Awareness)

TIDES (Translingual Information Detection Extraction Summarization)

Human Language researches computer systems that understand and/or synthesize spoken and written human languages. Included in this area are speech processing (recognition, understanding, and synthesis), information extraction, handwriting recognition, machine translation, text summarization, and language generation.

Babylon

DARPA Office: IAO
DARPA PM: LTC James Bass

Lisa Harper, Principal Investigator

Washington

Problem
Commercial enterprise does not support translation of languages spoken by small populations, even when there is high probability that terrorists use these languages. Ground troops and support personnel have limited access to human translation support when deployed to regions of the world where these languages are spoken.

Objectives
The goal of the Babylon program is to develop rapid, two-way, natural-language speech translation interfaces and platforms for warfighters to use in field environments for force protection, refugee processing, and medical triage. Babylon will focus on overcoming the many technical and engineering challenges limiting current multilingual translation technology to enable future full-domain, unconstrained dialogue translation in multiple environments.

Activities
MITRE is responsible for evaluating 1+1-way systems by June of FY03 and two-way translation systems by early FY04. MITRE will help develop new metrics to account for the unique characteristics of speech-to-speech translation systems. In support of these evaluations, MITRE is also spearheading the collection of foreign language dialogue data that will reflect end-user requirements.

Impacts
Babylon translation devices will aid personnel with Special Forces missions, emergency medical response, intelligence gathering, and force protection. Critically, Babylon targets low-density languages such as Pashto, in addition to other high-priority languages such as Farsi, Mandarin Chinese, and Arabic.

Core Dialogue Research

Christy Doran, Principal Investigator

Bedford and Washington

Problem
Dialogue managers (DMs) are of increasing interest to our sponsors, but have not been useful to date because they are not flexible enough to handle conversations of moderate complexity, multiple modalities, or more than two participants, or to be adapted to new conversational tasks or domains without considerable effort.

Objective
Our objectives are twofold: first, to advance the state of the art of operational DMs along the continuum of dialogue complexity, and, second, to develop a new paradigm for the rapid development of modular, extensible and robust DMs and for their evaluation.

Activities
In year one, we will assess the dialogue needs of three areas – training, question-answering, and multimodal, multiparty robot control – by porting existing DMs to them. In year two, we will focus on developing our modular information-state DM toolkit. In year three, we will formally evaluate our development paradigm by porting our toolkit to the same three areas.

Impact
By promoting a systematic approach to development of robust, portable DMs, we will transition this technology out of the laboratory into sponsor hands. The experience we gain in evaluating the portability and robustness of the toolkit will give sponsors the information they need to evaluate the potential effort and resources needed to build a new dialogue system.

Foreign Language Tool Improvement Through Evaluation — FLITE

Florence Reeder, Principal Investigator

Washington

Problem
Foreign language processing problems have not diminished. Instead of less data and fewer languages, we see more. Instead of more analysts and linguists, there are fewer. The need for better foreign language processing and translation tools is more critical and the penalties for failure are more spectacular. We seek to improve systems and their capabilities.

Objectives
We will establish automated evaluation of language processing systems for multiple language tools. We will start with automated evaluation for machine translation (MT) tools, use these to select between alternate translations, and use this framework to improve the quality of language generated in MT systems. Our tools will incorporate recent advances in natural language generation (NLG).

Activities
We will define core evaluation criteria for NLG, both stand-alone and integrated with MT. We will use multiple evaluation strategies and techniques, examine existing metrics in light of what they can / cannot measure, and apply evaluation techniques to two problems. We will then isolate portions that allow evaluation to choose between alternate translations and use evaluations in machine learning-based algorithms.

Impacts
This work addresses a shortfall in NLG. It supports on-demand evaluations for multiple customers, and also supports "MT in a box" for adaptation of MT to specific domains.

Reading Comprehension: Reading, Learning, Teaching

Lynette Hirschman, Principal Investigator

Bedford and Washington

Problem
This project is addressing a three-stage grand challenge application for human language technology: building a system that can “learn to read,” then “read to learn,” and finally “teach to learn.” It deals with issues of machine learning, knowledge acquisition, and instructional technology.

Objectives
First we will build a computer-based system capable of passing a third grade reading-comprehension test. Second we will build a system that will "read to learn," passing a test on that subject matter after having read the text. Finally we will build a system that can learn through interacting with a person, and, at the same time, help to teach the person.

Activities
We have applied prototype systems on reading comprehension tests designed for fourth to eighth graders with a 30%–40% accuracy. We are improving the system to include more components. We will implement a reciprocal teaching demonstration, where the system plays the role of teacher (grading student answers) or the role of peer learner (answering questions posed by a real student).

Impacts
This research will open new areas of research, addressing issues of machine learning, breaking the knowledge acquisition bottleneck, developing new evaluation measures for understanding and learning, and creating new instructional technologies via learning companions and interactive teaching environments.

TIA (Total Information Awareness)

Inderjeet Mani, Principal Investigator

Washington

TIDES (Translingual Information Detection Extraction Summarization)

DARPA Office: IAO
DARPA PM: Mr. Charles Wayne

Laurie Damianos, Principal Investigator

Bedford and Washington

Problem
Over the years, expanded trade and travel have increased the potential economic and political impacts of major disease outbreaks. Recently, biological terrorism has become a very real threat. Appropriate response to disease outbreaks and emerging threats depends on obtaining reliable and up-to-date information, which often means monitoring many news sources, particularly local news sources, in many languages worldwide.

Objectives
TIDES (Translingual Information Detection Extraction Summarization) aims to revolutionize the way that information is obtained from human language by enabling people to find and interpret needed information quickly and effectively, regardless of language or medium.

Activities
MITRE's role in the TIDES Integrated Feasibility Experiment-Translingual will cover data collection/processing/distribution, machine translation, and data exchange standards. We will also support the Total Information Awareness program through integration and training, geospatial and temporal normalization, topic tracking, and rapid domain portability. We will provide a test bed for research and continue to maintain and improve the operational MITRE Text and Audio Processing (MiTAP) systems.

Impacts
MiTAP has been deployed to four sites. It is being used to track global threats, including disease outbreaks and terrorist activity. MiTAP focuses on providing timely multilingual, global information access to over 450 analysts, medical experts, government users, and humanitarian organizations. A MiTAP product, the World Press Update, is distributed to hundreds of readers and decision-makers worldwide.