Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference

May 06, 2018
Adam Poliak, Yonatan Belinkov, James Glass, Benjamin Van Durme

Share this with someone who'll enjoy it:

We propose a process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena. We use these representations as features to train a natural language inference (NLI) classifier based on datasets recast from existing semantic annotations. In applying this process to a representative NMT system, we find its encoder appears most suited to supporting inferences at the syntax-semantics interface, as compared to anaphora resolution requiring world-knowledge. We conclude with a discussion on the merits and potential deficiencies of the existing process, and how it may be improved and extended as a broader framework for evaluating semantic coverage.

* To be presented at NAACL 2018 - 11 pages 

   Access Paper Source

Share this with someone who'll enjoy it: