Alert button

On Event Individuation for Document-Level Information Extraction

Add code
Alert button
Dec 19, 2022
Figure 1 for On Event Individuation for Document-Level Information Extraction
Figure 2 for On Event Individuation for Document-Level Information Extraction
Figure 3 for On Event Individuation for Document-Level Information Extraction
Figure 4 for On Event Individuation for Document-Level Information Extraction

Share this with someone who'll enjoy it:

As information extraction (IE) systems have grown more capable at whole-document extraction, the classic task of \emph{template filling} has seen renewed interest as a benchmark for evaluating them. In this position paper, we call into question the suitability of template filling for this purpose. We argue that the task demands definitive answers to thorny questions of \emph{event individuation} -- the problem of distinguishing distinct events -- about which even human experts disagree. We show through annotation studies and error analysis that this raises concerns about the usefulness of template filling evaluation metrics, the quality of datasets for the task, and the ability of models to learn it. Finally, we consider possible solutions.

Share this with someone who'll enjoy it: