Welcome to The MITRE Corporation's free online evaluation tool for the research and government community.
The MITRE Challenge site, launched in January 2011 to support Challenge #1: Multicultural Person Name Matching, provides a forum for academic institutions, commercial companies, government laboratories, and individuals to put their name matching systems to the test—and to receive immediate feedback. While Challenge #1 is no longer an active competition, The MITRE Challenge Squad reopened the site in May 2012 to serve as an open evaluation resource.
Challenge Site Reopened for the Benefit of the Research Community
By popular demand, The MITRE Challenge Squad reopened the Challenge site in May 2012 as a freely available online evaluation resource for the research community and for government organizations that wish to evaluate their name matching systems and test their ideas for system improvement.
How Did the Challenge Begin?
In January 2011, MITRE launched the first in a series of open challenges to encourage innovation in technologies of interest to the federal government. Challenge #1 entailed multicultural name matching—a technology that is a key component of identity matching, which involves measuring the similarity of database records referring to people. Uses include verifying eligibility for Social Security or medical benefits, identifying and reunifying families in disaster relief operations, vetting persons against a travel watchlist, and merging or eliminating duplicate records in databases. Person name matching can also be used to improve the accuracy and speed of document searches, social network analysis, and other tasks in which the same person might be referred to by multiple versions or spellings of a name.
The task was to match a query file and an index file, each containing a list of names, against one another and produce a list of scored matches for each query name. Participants received a dataset and task guidelines, submitted responses, and received immediate feedback on their performance. A continuously updated leaderboard let Challenge participants (whose names were kept anonymous) know where they stood against their competitors.
The first Challenge concluded on September 7, 2011, and the winners were announced at the Challenge TEM on Wednesday, October 5, in McLean, Va. The top 10 performers included corporate teams from Washington, D.C. and the Netherlands, graduate students from Washington and Georgia, and a team of Lebanese engineers.
 |
Team Name Selection
Participants choose the name of their team. The names of the best performing teams will be posted on a continuously updated leaderboard.
Timeframe
The MITRE Challenge launched in winter 2011 and closed on September 7, 2011, and the Challenge site was reopened in May 2012 as a freely available online evaluation resource for the research community and for government organizations that wish to evaluate their name matching systems and test their ideas for system improvement.
Contact
For more information, contact mitrechallenge@mitre.org.
General Terms and Conditions
Click to learn more about the Terms and Conditions of the MITRE Challenge competition.
|
|
Take the Challenge! |
User Benefits
- A realistic dataset, which can be used for system verification.
- Objective evaluation metrics, including F measure, F measure weighted for precision and for recall, and Mean Average Precision (MAP), to guide system improvements.
- Immediate feedback: results are automatically verified and scored.
About The MITRE Corporation
The MITRE Corporation is a not-for-profit organization that provides systems engineering, research and development, and information technology support to the government. It operates federally funded research and development centers (FFRDCs) for the Department of Defense, the Federal Aviation Administration, the Internal Revenue Service and Department of Veterans Affairs, the Department of Homeland Security, and the Administrative Office of the U.S. Courts, with principal locations in Bedford, Mass., and McLean, Va.
As an operator of FFRDCs that support various government agencies, MITRE maintains an objective, independent point of view with regard to industry in order to make unbiased recommendations to its U.S. government sponsors. FFRDCs support their sponsors across a full spectrum of planning and concept development, R&D, and systems acquisition. To ensure the highest levels of objectivity, FFRDCs do not manufacture products, compete with industry, or work for commercial companies.
|
|
