Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox

A Review of Evaluation Techniques for Social Dialogue Systems

Add code

Sep 13, 2017
Amanda Cercas Curry, Helen Hastie, Verena Rieser

Share this with someone who'll enjoy it:

In contrast with goal-oriented dialogue, social dialogue has no clear measure of task success. Consequently, evaluation of these systems is notoriously hard. In this paper, we review current evaluation methods, focusing on automatic metrics. We conclude that turn-based metrics often ignore the context and do not account for the fact that several replies are valid, while end-of-dialogue rewards are mainly hand-crafted. Both lack grounding in human perceptions.

* 2 pages 

   Access Paper Source

Share this with someone who'll enjoy it: