Journal Article

À la Recherche du Temps Perdu: extracting temporal relations from medical text in the 2012 i2b2 NLP challenge

Colin Cherry, Xiaodan Zhu, Joel Martin and Berry de Bruijn

in Journal of the American Medical Informatics Association

Published on behalf of American Medical Informatics Association

Volume 20, issue 5, pages 843-848
Published in print September 2013 | ISSN: 1067-5027
Published online March 2013 | e-ISSN: 1527-974X | DOI:

More Like This

Show all results sharing these subjects:

  • Medical Statistics and Methodology
  • Bioinformatics and Computational Biology
  • Biomathematics and Statistics


Show Summary Details



Objective An analysis of the timing of events is critical for a deeper understanding of the course of events within a patient record. The 2012 i2b2 NLP challenge focused on the extraction of temporal relationships between concepts within textual hospital discharge summaries.

Materials and methods The team from the National Research Council Canada (NRC) submitted three system runs to the second track of the challenge: typifying the time-relationship between pre-annotated entities. The NRC system was designed around four specialist modules containing statistical machine learning classifiers. Each specialist targeted distinct sets of relationships: local relationships, ‘sectime’-type relationships, non-local overlap-type relationships, and non-local causal relationships.

Results The best NRC submission achieved a precision of 0.7499, a recall of 0.6431, and an F1 score of 0.6924, resulting in a statistical tie for first place. Post hoc improvements led to a precision of 0.7537, a recall of 0.6455, and an F1 score of 0.6954, giving the highest scores reported on this task to date.

Discussion and conclusions Methods for general relation extraction extended well to temporal relations, and gave top-ranked state-of-the-art results. Careful ordering of predictions within result sets proved critical to this success.

Keywords: information extraction; temporal reasoning; natural language processing; relation extraction; clinical text

Journal Article.  5027 words.  Illustrated.

Subjects: Medical Statistics and Methodology ; Bioinformatics and Computational Biology ; Biomathematics and Statistics

Users without a subscription are not able to see the full content. Please, subscribe or login to access all content. subscribe or login to access all content.