Negation's Not Solved: Reconsidering Negation Annotation and Evaluation

October 2013
Stephen Wu, Department of Health Sciences Research, Mayo Clinic
James Masanz, Department of Health Sciences Research, Mayo Clinic
Matt Coarr, The MITRE Corporation
Scott Halgrim, Group Health Research Institute
David Carrell, Group Health Research Institute
Cheryl Clark, The MITRE Corporation
Timothy Miller, Children's Hospital Boston Informatics Program, Harvard Medical School
Download PDF (268.04 KB)

Though negation detection is a straightforward task in relatively constrained settings, when evaluated in heterogeneous corpora and annotation schema, performance may fall well below that of published benchmarks for both machine learning-based and rule-based approaches. Furthermore, it is difficult to determine the optimal mix of training data, or a standardized way to constrain evaluation metrics, since both are influenced by the corpus and annotation characteristics.

Publications

Interested in MITRE's Work?

MITRE provides affordable, effective solutions that help the government meet its most complex challenges.
Explore Job Openings

Publication Search