Alert button
Picture for Ayal Klein

Ayal Klein

Alert button

QASem Parsing: Text-to-text Modeling of QA-based Semantics

May 23, 2022
Ayal Klein, Eran Hirsch, Ron Eliav, Valentina Pyatkin, Avi Caciularu, Ido Dagan

Figure 1 for QASem Parsing: Text-to-text Modeling of QA-based Semantics
Figure 2 for QASem Parsing: Text-to-text Modeling of QA-based Semantics
Figure 3 for QASem Parsing: Text-to-text Modeling of QA-based Semantics
Figure 4 for QASem Parsing: Text-to-text Modeling of QA-based Semantics

Several recent works have suggested to represent semantic relations with questions and answers, decomposing textual information into separate interrogative natural language statements. In this paper, we consider three QA-based semantic tasks - namely, QA-SRL, QANom and QADiscourse, each targeting a certain type of predication - and propose to regard them as jointly providing a comprehensive representation of textual information. To promote this goal, we investigate how to best utilize the power of sequence-to-sequence (seq2seq) pre-trained language models, within the unique setup of semi-structured outputs, consisting of an unordered set of question-answer pairs. We examine different input and output linearization strategies, and assess the effect of multitask learning and of simple data augmentation techniques in the setting of imbalanced training data. Consequently, we release the first unified QASem parsing tool, practical for downstream applications who can benefit from an explicit, QA-based account of information units in a text.

Viaarxiv icon

QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions

Sep 26, 2021
Daniela Brook Weiss, Paul Roit, Ayal Klein, Ori Ernst, Ido Dagan

Figure 1 for QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions
Figure 2 for QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions
Figure 3 for QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions
Figure 4 for QA-Align: Representing Cross-Text Content Overlap by Aligning Question-Answer Propositions

Multi-text applications, such as multi-document summarization, are typically required to model redundancies across related texts. Current methods confronting consolidation struggle to fuse overlapping information. In order to explicitly represent content overlap, we propose to align predicate-argument relations across texts, providing a potential scaffold for information consolidation. We go beyond clustering coreferring mentions, and instead model overlap with respect to redundancy at a propositional level, rather than merely detecting shared referents. Our setting exploits QA-SRL, utilizing question-answer pairs to capture predicate-argument relations, facilitating laymen annotation of cross-text alignments. We employ crowd-workers for constructing a dataset of QA-based alignments, and present a baseline QA alignment model trained over our dataset. Analyses show that our new task is semantically challenging, capturing content overlap beyond lexical similarity and complements cross-document coreference with proposition-level links, offering potential use for downstream tasks.

* Accepted to EMNLP 2021, Main Conference 
Viaarxiv icon

QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines

Oct 06, 2020
Valentina Pyatkin, Ayal Klein, Reut Tsarfaty, Ido Dagan

Figure 1 for QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines
Figure 2 for QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines
Figure 3 for QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines
Figure 4 for QADiscourse -- Discourse Relations as QA Pairs: Representation, Crowdsourcing and Baselines

Discourse relations describe how two propositions relate to one another, and identifying them automatically is an integral part of natural language understanding. However, annotating discourse relations typically requires expert annotators. Recently, different semantic aspects of a sentence have been represented and crowd-sourced via question-and-answer (QA) pairs. This paper proposes a novel representation of discourse relations as QA pairs, which in turn allows us to crowd-source wide-coverage data annotated with discourse relations, via an intuitively appealing interface for composing such questions and answers. Based on our proposed representation, we collect a novel and wide-coverage QADiscourse dataset, and present baseline algorithms for predicting QADiscourse relations.

* To appear at EMNLP 2020 
Viaarxiv icon

Crowdsourcing a High-Quality Gold Standard for QA-SRL

Nov 08, 2019
Paul Roit, Ayal Klein, Daniela Stepanov, Jonathan Mamou, Julian Michael, Gabriel Stanovsky, Luke Zettlemoyer, Ido Dagan

Figure 1 for Crowdsourcing a High-Quality Gold Standard for QA-SRL
Figure 2 for Crowdsourcing a High-Quality Gold Standard for QA-SRL
Figure 3 for Crowdsourcing a High-Quality Gold Standard for QA-SRL
Figure 4 for Crowdsourcing a High-Quality Gold Standard for QA-SRL

Question-answer driven Semantic Role Labeling (QA-SRL) has been proposed as an attractive open and natural form of SRL, easily crowdsourceable for new corpora. Recently, a large-scale QA-SRL corpus and a trained parser were released, accompanied by a densely annotated dataset for evaluation. Trying to replicate the QA-SRL annotation and evaluation scheme for new texts, we observed that the resulting annotations were lacking in quality and coverage, particularly insufficient for creating gold standards for evaluation. In this paper, we present an improved QA-SRL annotation protocol, involving crowd-worker selection and training, followed by data consolidation. Applying this process, we release a new gold evaluation dataset for QA-SRL, yielding more consistent annotations and greater coverage. We believe that our new annotation protocol and gold standard will facilitate future replicable research of natural semantic annotations.

Viaarxiv icon