Alert button
Picture for Ludwig Sickert

Ludwig Sickert

Alert button

Inseq: An Interpretability Toolkit for Sequence Generation Models

Feb 27, 2023
Gabriele Sarti, Nils Feldhus, Ludwig Sickert, Oskar van der Wal

Figure 1 for Inseq: An Interpretability Toolkit for Sequence Generation Models
Figure 2 for Inseq: An Interpretability Toolkit for Sequence Generation Models
Figure 3 for Inseq: An Interpretability Toolkit for Sequence Generation Models
Figure 4 for Inseq: An Interpretability Toolkit for Sequence Generation Models

Past work in natural language processing interpretability focused mainly on popular classification tasks while largely overlooking generation settings, partly due to a lack of dedicated tools. In this work, we introduce Inseq, a Python library to democratize access to interpretability analyses of sequence generation models. Inseq enables intuitive and optimized extraction of models' internal information and feature importance scores for popular decoder-only and encoder-decoder Transformers architectures. We showcase its potential by adopting it to highlight gender biases in machine translation models and locate factual knowledge inside GPT-2. Thanks to its extensible interface supporting cutting-edge techniques such as contrastive feature attribution, Inseq can drive future advances in explainable natural language generation, centralizing good practices and enabling fair and reproducible model evaluations.

* Library: https://github.com/inseq-team/inseq, Documentation: https://inseq.readthedocs.io, v0.4 
Viaarxiv icon