Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

"Topic": models, code, and papers

Multi-class Hierarchical Question Classification for Multiple Choice Science Exams

Aug 15, 2019
Dongfang Xu, Peter Jansen, Jaycie Martin, Zhengnan Xie, Vikas Yadav, Harish Tayyar Madabushi, Oyvind Tafjord, Peter Clark

Prior work has demonstrated that question classification (QC), recognizing the problem domain of a question, can help answer it more accurately. However, developing strong QC algorithms has been hindered by the limited size and complexity of annotated data available. To address this, we present the largest challenge dataset for QC, containing 7,787 science exam questions paired with detailed classification labels from a fine-grained hierarchical taxonomy of 406 problem domains. We then show that a BERT-based model trained on this dataset achieves a large (+0.12 MAP) gain compared with previous methods, while also achieving state-of-the-art performance on benchmark open-domain and biomedical QC datasets. Finally, we show that using this model's predictions of question topic significantly improves the accuracy of a question answering system by +1.7% [email protected], with substantial future gains possible as QC performance improves.

  Access Paper or Ask Questions

Topological Data Analysis of Time Series Data for B2B Customer Relationshop Management

May 26, 2019
Rodrigo Rivera-Castro, Polina Pilyugina, Alexander Pletnev, Ivan Maksimov, Wanyi Wyz, Evgeny Burnaev

Topological Data Analysis (TDA) is a recent approach to analyze data sets from the perspective of their topological structure. Its use for time series data has been limited to the field of financial time series primarily and as a method for feature generation in machine learning applications. In this work, TDA is presented as a technique to gain additional understanding of the customers' loyalty for business-to-business customer relationship management. Increasing loyalty and strengthening relationships with key accounts remain an active topic of discussion both for researchers and managers. Using two public and two proprietary data sets of commercial data, this research shows that the technique enables analysts to better understand their customer base and identify prospective opportunities. In addition, the approach can be used as a clustering method to increase the accuracy of a predictive model for loyalty scoring. This work thus seeks to introduce TDA as a viable tool for data analysis to the quantitate marketing practitioner.

* Industrial Marketing & Purchasing Group Conference (IMP19), 2019 
* 9 pages, 2 figures, 1 table 

  Access Paper or Ask Questions

A Simple Explanation for the Existence of Adversarial Examples with Small Hamming Distance

Jan 30, 2019
Adi Shamir, Itay Safran, Eyal Ronen, Orr Dunkelman

The existence of adversarial examples in which an imperceptible change in the input can fool well trained neural networks was experimentally discovered by Szegedy et al in 2013, who called them "Intriguing properties of neural networks". Since then, this topic had become one of the hottest research areas within machine learning, but the ease with which we can switch between any two decisions in targeted attacks is still far from being understood, and in particular it is not clear which parameters determine the number of input coordinates we have to change in order to mislead the network. In this paper we develop a simple mathematical framework which enables us to think about this baffling phenomenon from a fresh perspective, turning it into a natural consequence of the geometry of $\mathbb{R}^n$ with the $L_0$ (Hamming) metric, which can be quantitatively analyzed. In particular, we explain why we should expect to find targeted adversarial examples with Hamming distance of roughly $m$ in arbitrarily deep neural networks which are designed to distinguish between $m$ input classes.

  Access Paper or Ask Questions

Challenges in Designing Datasets and Validation for Autonomous Driving

Jan 26, 2019
Michal Uricar, David Hurych, Pavel Krizek, Senthil Yogamani

Autonomous driving is getting a lot of attention in the last decade and will be the hot topic at least until the first successful certification of a car with Level 5 autonomy. There are many public datasets in the academic community. However, they are far away from what a robust industrial production system needs. There is a large gap between academic and industrial setting and a substantial way from a research prototype, built on public datasets, to a deployable solution which is a challenging task. In this paper, we focus on bad practices that often happen in the autonomous driving from an industrial deployment perspective. Data design deserves at least the same amount of attention as the model design. There is very little attention paid to these issues in the scientific community, and we hope this paper encourages better formalization of dataset design. More specifically, we focus on the datasets design and validation scheme for autonomous driving, where we would like to highlight the common problems, wrong assumptions, and steps towards avoiding them, as well as some open problems.

* Accepted at VISAPP 2019 

  Access Paper or Ask Questions

Security Matters: A Survey on Adversarial Machine Learning

Oct 23, 2018
Guofu Li, Pengjia Zhu, Jin Li, Zhemin Yang, Ning Cao, Zhiyi Chen

Adversarial machine learning is a fast growing research area, which considers the scenarios when machine learning systems may face potential adversarial attackers, who intentionally synthesize input data to make a well-trained model to make mistake. It always involves a defending side, usually a classifier, and an attacking side that aims to cause incorrect output. The earliest studies on the adversarial examples for machine learning algorithms start from the information security area, which considers a much wider varieties of attacking methods. But recent research focus that popularized by the deep learning community places strong emphasis on how the "imperceivable" perturbations on the normal inputs may cause dramatic mistakes by the deep learning with supposed super-human accuracy. This paper serves to give a comprehensive introduction to a range of aspects of the adversarial deep learning topic, including its foundations, typical attacking and defending strategies, and some extended studies.

  Access Paper or Ask Questions

Dynamics Estimation Using Recurrent Neural Network

Sep 17, 2018
Astha Sharma

There is a plenty of research going on in field of robotics. One of the most important task is dynamic estimation of response during motion. One of the main applications of this research topics is the task of pouring, which is performed daily and is commonly used while cooking. We present an approach to estimate response to a sequence of manipulation actions. We are experimenting with pouring motion and the response is the change of the amount of water in the pouring cup. The pouring motion is represented by rotation angle and the amount of water is represented by its weight. We are using recurrent neural networks for building the neural network model to train on sequences which represents 1307 trails of pouring. The model gives great results on unseen test data which does not too different with training data in terms of dimensions of the cup used for pouring and receiving. The loss obtained with this test data is 4.5920. The model does not give good results on generalization experiments when we provide a test set which has dimensions of the cup very different from those in training data.

  Access Paper or Ask Questions

Emergence of Communication in an Interactive World with Consistent Speakers

Sep 03, 2018
Ben Bogin, Mor Geva, Jonathan Berant

Training agents to communicate with one another given task-based supervision only has attracted considerable attention recently, due to the growing interest in developing models for human-agent interaction. Prior work on the topic focused on simple environments, where training using policy gradient was feasible despite the non-stationarity of the agents during training. In this paper, we present a more challenging environment for testing the emergence of communication from raw pixels, where training using policy gradient fails. We propose a new model and training algorithm, that utilizes the structure of a learned representation space to produce more consistent speakers at the initial phases of training, which stabilizes learning. We empirically show that our algorithm substantially improves performance compared to policy gradient. We also propose a new alignment-based metric for measuring context-independence in emerged communication and find our method increases context-independence compared to policy gradient and other competitive baselines.

  Access Paper or Ask Questions

Deciding the status of controversial phonemes using frequency distributions; an application to semiconsonants in Spanish

Aug 22, 2018
Manuel Ortega-Rodríguez, Hugo Solís-Sánchez, Ricardo Gamboa-Alfaro

Exploiting the fact that natural languages are complex systems, the present exploratory article proposes a direct method based on frequency distributions that may be useful when making a decision on the status of problematic phonemes, an open problem in linguistics. The main notion is that natural languages, which can be considered from a complex outlook as information processing machines, and which somehow manage to set appropriate levels of redundancy, already "made the choice" whether a linguistic unit is a phoneme or not, and this would be reflected in a greater smoothness in a frequency versus rank graph. For the particular case we chose to study, we conclude that it is reasonable to consider the Spanish semiconsonant /w/ as a separate phoneme from its vowel counterpart /u/, on the one hand, and possibly also the semiconsonant /j/ as a separate phoneme from its vowel counterpart /i/, on the other. As language has been so central a topic in the study of complexity, this discussion grants us, in addition, an opportunity to gain insight into emerging properties in the broader complex systems debate.

* Physica A 503 (2018) 1020-1029 
* 24 pages, 10 figures 

  Access Paper or Ask Questions

Unsupervised robust nonparametric learning of hidden community properties

Jul 23, 2018
Mikhail A. Langovoy, Akhilesh Gotmare, Martin Jaggi

We consider learning of fundamental properties of communities in large noisy networks, in the prototypical situation where the nodes or users are split into two classes according to a binary property, e.g., according to their opinions or preferences on a topic. For learning these properties, we propose a nonparametric, unsupervised, and scalable graph scan procedure that is, in addition, robust against a class of powerful adversaries. In our setup, one of the communities can fall under the influence of a knowledgeable adversarial leader, who knows the full network structure, has unlimited computational resources and can completely foresee our planned actions on the network. We prove strong consistency of our results in this setup with minimal assumptions. In particular, the learning procedure estimates the baseline activity of normal users asymptotically correctly with probability 1; the only assumption being the existence of a single implicit community of asymptotically negligible logarithmic size. We provide experiments on real and synthetic data to illustrate the performance of our method, including examples with adversaries.

* Experiments with new types of adversaries added 

  Access Paper or Ask Questions

Closed-Loop Policies for Operational Tests of Safety-Critical Systems

May 19, 2018
Jeremy Morton, Tim A. Wheeler, Mykel J. Kochenderfer

Manufacturers of safety-critical systems must make the case that their product is sufficiently safe for public deployment. Much of this case often relies upon critical event outcomes from real-world testing, requiring manufacturers to be strategic about how they allocate testing resources in order to maximize their chances of demonstrating system safety. This work frames the partially observable and belief-dependent problem of test scheduling as a Markov decision process, which can be solved efficiently to yield closed-loop manufacturer testing policies. By solving for policies over a wide range of problem formulations, we are able to provide high-level guidance for manufacturers and regulators on issues relating to the testing of safety-critical systems. This guidance spans an array of topics, including circumstances under which manufacturers should continue testing despite observed incidents, when manufacturers should test aggressively, and when regulators should increase or reduce the real-world testing requirements for an autonomous vehicle.

* 12 pages, 5 figures, 5 tables 

  Access Paper or Ask Questions