Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

Three New Methods for Evaluating Reference Resolution

Add code

Aug 21, 2002
Andrei Popescu-Belis, Isabelle Robba

Share this with someone who'll enjoy it:

Reference resolution on extended texts (several thousand references) cannot be evaluated manually. An evaluation algorithm has been proposed for the MUC tests, using equivalence classes for the coreference relation. However, we show here that this algorithm is too indulgent, yielding good scores even for poor resolution strategies. We elaborate on the same formalism to propose two new evaluation algorithms, comparing them first with the MUC algorithm and giving then results on a variety of examples. A third algorithm using only distributional comparison of equivalence classes is finally described; it assesses the relative importance of the recall vs. precision errors.

* Proceedings of the LREC'98 Workshop on Linguistic Coreference, Madrid, Spain, 1998 
* 7 pages 

   Access Paper Source

Share this with someone who'll enjoy it: