Alert button

A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets

Jun 21, 2020
Chengchang Zeng, Shaobo Li, Qin Li, Jie Hu, Jianjun Hu

Figure 1 for A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets
Figure 2 for A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets
Figure 3 for A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets
Figure 4 for A Survey on Machine Reading Comprehension: Tasks, Evaluation Metrics, and Benchmark Datasets

Share this with someone who'll enjoy it:

Machine Reading Comprehension (MRC) is a challenging NLP research field with wide real world applications. The great progress of this field in recent years is mainly due to the emergence of large-scale datasets and deep learning. At present, a lot of MRC models have already surpassed the human performance on many datasets despite the obvious giant gap between existing MRC models and genuine human-level reading comprehension. This shows the need of improving existing datasets, evaluation metrics and models to move the MRC models toward 'real' understanding. To address this lack of comprehensive survey of existing MRC tasks, evaluation metrics and datasets, herein, (1) we analyzed 57 MRC tasks and datasets; proposed a more precise classification method of MRC tasks with 4 different attributes (2) we summarized 9 evaluation metrics of MRC tasks and (3) 7 attributes and 10 characteristics of MRC datasets; (4) We also discussed some open issues in MRC research and highlight some future research directions. In addition, to help the community, we have collected, organized, and published our data on a companion website(https://mrc-datasets.github.io/) where MRC researchers could directly access each MRC dataset, papers, baseline projects and browse the leaderboard.

* 59 pages  
View paper onarxiv icon

Share this with someone who'll enjoy it: