Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox


Towards Human-Centred Explainability Benchmarks For Text Classification

Add code

Nov 10, 2022
Viktor Schlegel, Erick Mendez-Guzman, Riza Batista-Navarro


Share this with someone who'll enjoy it:


Progress on many Natural Language Processing (NLP) tasks, such as text classification, is driven by objective, reproducible and scalable evaluation via publicly available benchmarks. However, these are not always representative of real-world scenarios where text classifiers are employed, such as sentiment analysis or misinformation detection. In this position paper, we put forward two points that aim to alleviate this problem. First, we propose to extend text classification benchmarks to evaluate the explainability of text classifiers. We review challenges associated with objectively evaluating the capabilities to produce valid explanations which leads us to the second main point: We propose to ground these benchmarks in human-centred applications, for example by using social media, gamification or to learn explainability metrics from human judgements.

* Accepted at NeatClass @ ICSWSM 2022 


   Access Paper Source



Share this with someone who'll enjoy it: