Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

"Topic": models, code, and papers

A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods

Apr 07, 2022
Zhihan Zhang, Wenhao Yu, Mengxia Yu, Zhichun Guo, Meng Jiang

Multi-task learning (MTL) has become increasingly popular in natural language processing (NLP) because it improves the performance of related tasks by exploiting their commonalities and differences. Nevertheless, it is still not understood very well how multi-task learning can be implemented based on the relatedness of training tasks. In this survey, we review recent advances of multi-task learning methods in NLP, with the aim of summarizing them into two general multi-task training methods based on their task relatedness: (i) joint training and (ii) multi-step training. We present examples in various NLP downstream applications, summarize the task relationships and discuss future directions of this promising topic.

  Access Paper or Ask Questions

Neural Forecasting of the Italian Sovereign Bond Market with Economic News

Mar 11, 2022
Sergio Consoli, Luca Tiozzo Pezzoli, Elisa Tosetti

In this paper we employ economic news within a neural network framework to forecast the Italian 10-year interest rate spread. We use a big, open-source, database known as Global Database of Events, Language and Tone to extract topical and emotional news content linked to bond markets dynamics. We deploy such information within a probabilistic forecasting framework with autoregressive recurrent networks (DeepAR). Our findings suggest that a deep learning network based on Long-Short Term Memory cells outperforms classical machine learning techniques and provides a forecasting performance that is over and above that obtained by using conventional determinants of interest rates alone.

* Journal of the Royal Statistical Society - Series A (Statistics in Society), 2022 
* 24 pages, 8 figures, in press 

  Access Paper or Ask Questions

Corpus for Automatic Structuring of Legal Documents

Jan 31, 2022
Prathamesh Kalamkar, Aman Tiwari, Astha Agarwal, Saurabh Karn, Smita Gupta, Vivek Raghavan, Ashutosh Modi

In populous countries, pending legal cases have been growing exponentially. There is a need for developing techniques for processing and organizing legal documents. In this paper, we introduce a new corpus for structuring legal documents. In particular, we introduce a corpus of legal judgment documents in English that are segmented into topical and coherent parts. Each of these parts is annotated with a label coming from a list of pre-defined Rhetorical Roles. We develop baseline models for automatically predicting rhetorical roles in a legal document based on the annotated corpus. Further, we show the application of rhetorical roles to improve performance on the tasks of summarization and legal judgment prediction. We release the corpus and baseline model code along with the paper.

* 10 Pages (8 page main paper + 2 page references) 

  Access Paper or Ask Questions

A Brief History of Updates of Answer-Set Programs

Dec 27, 2021
João Leite, Martin Slota

Over the last couple of decades, there has been a considerable effort devoted to the problem of updating logic programs under the stable model semantics (a.k.a. answer-set programs) or, in other words, the problem of characterising the result of bringing up-to-date a logic program when the world it describes changes. Whereas the state-of-the-art approaches are guided by the same basic intuitions and aspirations as belief updates in the context of classical logic, they build upon fundamentally different principles and methods, which have prevented a unifying framework that could embrace both belief and rule updates. In this paper, we will overview some of the main approaches and results related to answer-set programming updates, while pointing out some of the main challenges that research in this topic has faced.

* Under consideration in Theory and Practice of Logic Programming (TPLP) 

  Access Paper or Ask Questions

A Bibliometric Analysis of the BPM Conference Using Computational Data Analytics

Nov 18, 2021
Fabian Muff, Felix Härer, Hans-Georg Fill

The BPM conference has a long tradition as the premier venue for publishing research on business process management. For exploring the evolution of research topics, we present the findings from a computational bibliometric analysis of the BPM conference proceedings from the past 15 years. We used the publicly available DBLP dataset as a basis for the analysis, which we enriched with data from websites and databases of the relevant publishers. In addition to a detailed analysis of the publication results, we performed a content-based analysis of over 1,200 papers from the BPM conference and its workshops using Latent Dirichlet Allocation. This offers insights into historical developments in Business Process Management research and provides the community with potential future prospects.

  Access Paper or Ask Questions

Improving Large-scale Language Models and Resources for Filipino

Nov 11, 2021
Jan Christian Blaise Cruz, Charibeth Cheng

In this paper, we improve on existing language resources for the low-resource Filipino language in two ways. First, we outline the construction of the TLUnified dataset, a large-scale pretraining corpus that serves as an improvement over smaller existing pretraining datasets for the language in terms of scale and topic variety. Second, we pretrain new Transformer language models following the RoBERTa pretraining technique to supplant existing models trained with small corpora. Our new RoBERTa models show significant improvements over existing Filipino models in three benchmark datasets with an average gain of 4.47% test accuracy across the three classification tasks of varying difficulty.

* Resources are available at 

  Access Paper or Ask Questions

A Survey on Open Set Recognition

Aug 18, 2021
Atefeh Mahdavi, Marco Carvalho

Open Set Recognition (OSR) is about dealing with unknown situations that were not learned by the models during training. In this paper, we provide a survey of existing works about OSR and distinguish their respective advantages and disadvantages to help out new researchers interested in the subject. The categorization of OSR models is provided along with an extensive summary of recent progress. Additionally, the relationships between OSR and its related tasks including multi-class classification and novelty detection are analyzed. It is concluded that OSR can appropriately deal with unknown instances in the real-world where capturing all possible classes in the training data is not practical. Lastly, applications of OSR are highlighted and some new directions for future research topics are suggested.

* 17 pages, 6 figures 

  Access Paper or Ask Questions

An Analysis of the Recent Visibility of the SigDial Conference

Jun 30, 2021
Casey Kennington, McKenzie Steenson

Automated speech and text interfaces are continuing to improve, resulting in increased research in the area of dialogue systems. Moreover, conferences and workshops from various fields are focusing more on language through speech and text mediums as candidates for interaction with applications such as search interfaces and robots. In this paper, we explore how visible the SigDial conference is to outside conferences by analysing papers from top Natural Langauge Processing conferences since 2015 to determine the popularity of certain SigDial-related topics, as well as analysing what SigDial papers are being cited by others outside of SigDial. We find that despite a dramatic increase in dialogue-related research, SigDial visibility has not increased. We conclude by offering some suggestions.

  Access Paper or Ask Questions

Voxel Structure-based Mesh Reconstruction from a 3D Point Cloud

Apr 23, 2021
Chenlei Lv, Weisi Lin, Baoquan Zhao

Mesh reconstruction from a 3D point cloud is an important topic in the fields of computer graphic, computer vision, and multimedia analysis. In this paper, we propose a voxel structure-based mesh reconstruction framework. It provides the intrinsic metric to improve the accuracy of local region detection. Based on the detected local regions, an initial reconstructed mesh can be obtained. With the mesh optimization in our framework, the initial reconstructed mesh is optimized into an isotropic one with the important geometric features such as external and internal edges. The experimental results indicate that our framework shows great advantages over peer ones in terms of mesh quality, geometric feature keeping, and processing speed.

* Accepted by IEEE Transactions on Multimedia 

  Access Paper or Ask Questions

Theoretical Knowledge Graph Reasoning via Ending Anchored Rules

Dec 15, 2020
Canlin Zhang, Yannis Katsis, Yoshiki Vazquez-Baeza, Andrew Bartko, Ho-Cheol Kim, Chun-Nan Hsu

Discovering precise and specific rules from knowledge graphs is regarded as an essential challenge, which can improve the performances of many downstream tasks and even provide new ways to approach some Natural Language Processing research topics. In this paper, we provide a fundamental theory for knowledge graph reasoning based on the ending anchored rules. Our theory provides precise reasons explaining why or why not a triple is correct. Then, we implement our theory by what we call the EARDict model. Results show that our EARDict model significantly outperforms all the benchmark models on two large datasets of knowledge graph completion, including achieving a [email protected] score of 96.6 percent on WN18RR.

* Comparing to v2, v3 raises the lower bound of the connection set to be 2, which increases the performances on WN18RR for about 20 percent, and increases those on FB15K-237 for about 6 percent. People may refer to our presentation "EARDict_refinement" posted on for a detailed comparison between v2 and v3. We also revise our expressions a lot in v3 

  Access Paper or Ask Questions