Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

"Topic": models, code, and papers

Graph Embedding Techniques, Applications, and Performance: A Survey

Dec 22, 2017
Palash Goyal, Emilio Ferrara

Graphs, such as social networks, word co-occurrence networks, and communication networks, occur naturally in various real-world applications. Analyzing them yields insight into the structure of society, language, and different patterns of communication. Many approaches have been proposed to perform the analysis. Recently, methods which use the representation of graph nodes in vector space have gained traction from the research community. In this survey, we provide a comprehensive and structured analysis of various graph embedding techniques proposed in the literature. We first introduce the embedding task and its challenges such as scalability, choice of dimensionality, and features to be preserved, and their possible solutions. We then present three categories of approaches based on factorization methods, random walks, and deep learning, with examples of representative algorithms in each category and analysis of their performance on various tasks. We evaluate these state-of-the-art methods on a few common datasets and compare their performance against one another. Our analysis concludes by suggesting some potential applications and future directions. We finally present the open-source Python library we developed, named GEM (Graph Embedding Methods, available at, which provides all presented algorithms within a unified interface to foster and facilitate research on the topic.

* Knowledge Based Systems, Volume 151, 1 July 2018, Pages 78-94, 2018 
* Submitted to Knowledge Based Systems for review 

  Access Paper or Ask Questions

Kafnets: kernel-based non-parametric activation functions for neural networks

Nov 23, 2017
Simone Scardapane, Steven Van Vaerenbergh, Simone Totaro, Aurelio Uncini

Neural networks are generally built by interleaving (adaptable) linear layers with (fixed) nonlinear activation functions. To increase their flexibility, several authors have proposed methods for adapting the activation functions themselves, endowing them with varying degrees of flexibility. None of these approaches, however, have gained wide acceptance in practice, and research in this topic remains open. In this paper, we introduce a novel family of flexible activation functions that are based on an inexpensive kernel expansion at every neuron. Leveraging over several properties of kernel-based models, we propose multiple variations for designing and initializing these kernel activation functions (KAFs), including a multidimensional scheme allowing to nonlinearly combine information from different paths in the network. The resulting KAFs can approximate any mapping defined over a subset of the real line, either convex or nonconvex. Furthermore, they are smooth over their entire domain, linear in their parameters, and they can be regularized using any known scheme, including the use of $\ell_1$ penalties to enforce sparseness. To the best of our knowledge, no other known model satisfies all these properties simultaneously. In addition, we provide a relatively complete overview on alternative techniques for adapting the activation functions, which is currently lacking in the literature. A large set of experiments validates our proposal.

* Preprint submitted to Neural Networks (Elsevier) 

  Access Paper or Ask Questions

Ask the GRU: Multi-Task Learning for Deep Text Recommendations

Sep 09, 2016
Trapit Bansal, David Belanger, Andrew McCallum

In a variety of application domains the content to be recommended to users is associated with text. This includes research papers, movies with associated plot summaries, news articles, blog posts, etc. Recommendation approaches based on latent factor models can be extended naturally to leverage text by employing an explicit mapping from text to factors. This enables recommendations for new, unseen content, and may generalize better, since the factors for all items are produced by a compactly-parametrized model. Previous work has used topic models or averages of word embeddings for this mapping. In this paper we present a method leveraging deep recurrent neural networks to encode the text sequence into a latent vector, specifically gated recurrent units (GRUs) trained end-to-end on the collaborative filtering task. For the task of scientific paper recommendation, this yields models with significantly higher accuracy. In cold-start scenarios, we beat the previous state-of-the-art, all of which ignore word order. Performance is further improved by multi-task learning, where the text encoder network is trained for a combination of content recommendation and item metadata prediction. This regularizes the collaborative filtering model, ameliorating the problem of sparsity of the observed rating matrix.

* 8 pages 

  Access Paper or Ask Questions

Mining Arguments from Cancer Documents Using Natural Language Processing and Ontologies

Jul 27, 2016
Adrian Groza, Oana Popa

In the medical domain, the continuous stream of scientific research contains contradictory results supported by arguments and counter-arguments. As medical expertise occurs at different levels, part of the human agents have difficulties to face the huge amount of studies, but also to understand the reasons and pieces of evidences claimed by the proponents and the opponents of the debated topic. To better understand the supporting arguments for new findings related to current state of the art in the medical domain we need tools able to identify arguments in scientific papers. Our work here aims to fill the above technological gap. Quite aware of the difficulty of this task, we embark to this road by relying on the well-known interleaving of domain knowledge with natural language processing. To formalise the existing medical knowledge, we rely on ontologies. To structure the argumentation model we use also the expressivity and reasoning capabilities of Description Logics. To perform argumentation mining we formalise various linguistic patterns in a rule-based language. We tested our solution against a corpus of scientific papers related to breast cancer. The run experiments show a F-measure between 0.71 and 0.86 for identifying conclusions of an argument and between 0.65 and 0.86 for identifying premises of an argument.

* ICCP 2016 

  Access Paper or Ask Questions

Less is more: zero-shot learning from online textual documents with noise suppression

Apr 05, 2016
Ruizhi Qiao, Lingqiao Liu, Chunhua Shen, Anton van den Hengel

Classifying a visual concept merely from its associated online textual source, such as a Wikipedia article, is an attractive research topic in zero-shot learning because it alleviates the burden of manually collecting semantic attributes. Several recent works have pursued this approach by exploring various ways of connecting the visual and text domains. This paper revisits this idea by stepping further to consider one important factor: the textual representation is usually too noisy for the zero-shot learning application. This consideration motivates us to design a simple-but-effective zero-shot learning method capable of suppressing noise in the text. More specifically, we propose an $l_{2,1}$-norm based objective function which can simultaneously suppress the noisy signal in the text and learn a function to match the text document and visual features. We also develop an optimization algorithm to efficiently solve the resulting problem. By conducting experiments on two large datasets, we demonstrate that the proposed method significantly outperforms the competing methods which rely on online information sources but without explicit noise suppression. We further make an in-depth analysis of the proposed method and provide insight as to what kind of information in documents is useful for zero-shot learning.

* Accepted to Int. Conf. Computer Vision and Pattern Recognition (CVPR) 2016 

  Access Paper or Ask Questions

Collaborative Deep Learning for Recommender Systems

Jun 18, 2015
Hao Wang, Naiyan Wang, Dit-Yan Yeung

Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recent advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art.

  Access Paper or Ask Questions

A behavioural approach to obstacle avoidance for mobile manipulators based on distributed sensing

Feb 09, 2015
Luigi Palmieri

A reactive obstacle avoidance method for mobile manipulators is presented. The objectives of the developed algorithm are twofold. The first one is to find a trajectory in the configuration space of a mobile manipulator so as to follow a given trajectory in the task space. The second objective consists in locally adjusting the trajectory in the configuration space in order to avoid collisions with potentially moving obstacles and self-collisions in unstructured and dynamic environments. The perception is exclusively based on a set of proximity sensors distributed on the robot mechanical structure and visual information are not required. Thanks to the adoption of this kind of proximity distributed perception, the approach does not require a 3D model of the robot and allows the real-time collision avoidance without the need of a sensorized environment. To achieve the features cited above, a behaviour-based technique known as Null-Space-Based (NSB) approach has been adopted with some modifications.On one hand, the concept of a total pseudo-energy based on the information from the distributed sensors has been introduced. On the other hand, a method to combine different tasks has been proposed to guarantee the smoothness of the realtime trajectory adjustments. Another significant feature of the method is the strict coordination between the base and the arm exploiting the redundant degrees of freedom, that is a relevant topic in mobile manipulation.

* Master's Thesis 

  Access Paper or Ask Questions

Multiple Moving Object Recognitions in video based on Log Gabor-PCA Approach

Jul 03, 2014
M. T Gopalakrishna, M. Ravishankar, D. R Rameshbabu

Object recognition in the video sequence or images is one of the sub-field of computer vision. Moving object recognition from a video sequence is an appealing topic with applications in various areas such as airport safety, intrusion surveillance, video monitoring, intelligent highway, etc. Moving object recognition is the most challenging task in intelligent video surveillance system. In this regard, many techniques have been proposed based on different methods. Despite of its importance, moving object recognition in complex environments is still far from being completely solved for low resolution videos, foggy videos, and also dim video sequences. All in all, these make it necessary to develop exceedingly robust techniques. This paper introduces multiple moving object recognition in the video sequence based on LoG Gabor-PCA approach and Angle based distance Similarity measures techniques used to recognize the object as a human, vehicle etc. Number of experiments are conducted for indoor and outdoor video sequences of standard datasets and also our own collection of video sequences comprising of partial night vision video sequences. Experimental results show that our proposed approach achieves an excellent recognition rate. Results obtained are satisfactory and competent.

* 8,26,conference 

  Access Paper or Ask Questions

Quantifying Causal Coupling Strength: A Lag-specific Measure For Multivariate Time Series Related To Transfer Entropy

Nov 21, 2012
Jakob Runge, Jobst Heitzig, Norbert Marwan, Jürgen Kurths

While it is an important problem to identify the existence of causal associations between two components of a multivariate time series, a topic addressed in Runge et al. (2012), it is even more important to assess the strength of their association in a meaningful way. In the present article we focus on the problem of defining a meaningful coupling strength using information theoretic measures and demonstrate the short-comings of the well-known mutual information and transfer entropy. Instead, we propose a certain time-delayed conditional mutual information, the momentary information transfer (MIT), as a measure of association that is general, causal and lag-specific, reflects a well interpretable notion of coupling strength and is practically computable. MIT is based on the fundamental concept of source entropy, which we utilize to yield a notion of coupling strength that is, compared to mutual information and transfer entropy, well interpretable, in that for many cases it solely depends on the interaction of the two components at a certain lag. In particular, MIT is thus in many cases able to exclude the misleading influence of autodependency within a process in an information-theoretic way. We formalize and prove this idea analytically and numerically for a general class of nonlinear stochastic processes and illustrate the potential of MIT on climatological data.

* Physical Review E, 86, 061121 (2012) 
* 15 pages, 6 figures; accepted for publication in Physical Review E 

  Access Paper or Ask Questions

Latent Dirichlet Allocation Uncovers Spectral Characteristics of Drought Stressed Plants

Oct 16, 2012
Mirwaes Wahabzada, Kristian Kersting, Christian Bauckhage, Christoph Roemer, Agim Ballvora, Francisco Pinto, Uwe Rascher, Jens Leon, Lutz Ploemer

Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since it allows to discover non-destructively spectral characteristics of plants governed primarily by scattering and absorption characteristics of the leaf internal structure and biochemical constituents. Several drought stress indices have been derived using hyper-spectral imaging. However, they are typically based on few hyper-spectral images only, rely on interpretations of experts, and consider few wavelengths only. In this study, we present the first data-driven approach to discovering spectral drought stress indices, treating it as an unsupervised labeling problem at massive scale. To make use of short range dependencies of spectral wavelengths, we develop an online variational Bayes algorithm for latent Dirichlet allocation with convolved Dirichlet regularizer. This approach scales to massive datasets and, hence, provides a more objective complement to plant physiological practices. The spectral topics found conform to plant physiological knowledge and can be computed in a fraction of the time compared to existing LDA approaches.

* Appears in Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (UAI2012) 

  Access Paper or Ask Questions