Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

A Pragmatic Guide to Geoparsing Evaluation

Nov 05, 2018
Milan Gritta, Mohammad Taher Pilehvar, Nigel Collier

Share this with someone who'll enjoy it:

Empirical methods in geoparsing have thus far lacked a standard evaluation framework describing the task, data and metrics used to establish state-of-the-art systems. Evaluation is further made inconsistent, even unrepresentative of real-world usage, by the lack of distinction between the different types of toponyms, which necessitates new guidelines, a consolidation of metrics and a detailed toponym taxonomy with implications for Named Entity Recognition (NER). To address these deficiencies, our manuscript introduces such framework in three parts. Part 1) Task Definition: clarified via corpus linguistic analysis proposing a fine-grained Pragmatic Taxonomy of Toponyms with new guidelines. Part 2) Evaluation Data: shared via a dataset called GeoWebNews to provide test/train data to enable immediate use of our contributions. In addition to fine-grained Geotagging and Toponym Resolution (Geocoding), this dataset is also suitable for prototyping machine learning NLP models. Part 3) Metrics: discussed and reviewed for a rigorous evaluation with appropriate recommendations for NER/Geoparsing practitioners.

* Currently under review at Language Resources and Evaluation (Springer). We will update the pdf with the accepted version soon after the peer review. Thanks 

   Access Paper Source

Share this with someone who'll enjoy it: