Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

"Topic": models, code, and papers

Multi-Reference Alignment for sparse signals, Uniform Uncertainty Principles and the Beltway Problem

Jun 24, 2021
Subhro Ghosh, Philippe Rigollet

Motivated by cutting-edge applications like cryo-electron microscopy (cryo-EM), the Multi-Reference Alignment (MRA) model entails the learning of an unknown signal from repeated measurements of its images under the latent action of a group of isometries and additive noise of magnitude $\sigma$. Despite significant interest, a clear picture for understanding rates of estimation in this model has emerged only recently, particularly in the high-noise regime $\sigma \gg 1$ that is highly relevant in applications. Recent investigations have revealed a remarkable asymptotic sample complexity of order $\sigma^6$ for certain signals whose Fourier transforms have full support, in stark contrast to the traditional $\sigma^2$ that arise in regular models. Often prohibitively large in practice, these results have prompted the investigation of variations around the MRA model where better sample complexity may be achieved. In this paper, we show that \emph{sparse} signals exhibit an intermediate $\sigma^4$ sample complexity even in the classical MRA model. Our results explore and exploit connections of the MRA estimation problem with two classical topics in applied mathematics: the \textit{beltway problem} from combinatorial optimization, and \textit{uniform uncertainty principles} from harmonic analysis.

  Access Paper or Ask Questions

A Comprehensive Survey on Community Detection with Deep Learning

May 26, 2021
Xing Su, Shan Xue, Fanzhen Liu, Jia Wu, Jian Yang, Chuan Zhou, Wenbin Hu, Cecile Paris, Surya Nepal, Di Jin, Quan Z. Sheng, Philip S. Yu

A community reveals the features and connections of its members that are different from those in other communities in a network. Detecting communities is of great significance in network analysis. Despite the classical spectral clustering and statistical inference methods, we notice a significant development of deep learning techniques for community detection in recent years with their advantages in handling high dimensional network data. Hence, a comprehensive overview of community detection's latest progress through deep learning is timely to both academics and practitioners. This survey devises and proposes a new taxonomy covering different categories of the state-of-the-art methods, including deep learning-based models upon deep neural networks, deep nonnegative matrix factorization and deep sparse filtering. The main category, i.e., deep neural networks, is further divided into convolutional networks, graph attention networks, generative adversarial networks and autoencoders. The survey also summarizes the popular benchmark data sets, model evaluation metrics, and open-source implementations to address experimentation settings. We then discuss the practical applications of community detection in various domains and point to implementation scenarios. Finally, we outline future directions by suggesting challenging topics in this fast-growing deep learning field.

  Access Paper or Ask Questions

Robust Sample Weighting to Facilitate Individualized Treatment Rule Learning for a Target Population

May 03, 2021
Rui Chen, Jared D. Huling, Guanhua Chen, Menggang Yu

Learning individualized treatment rules (ITRs) is an important topic in precision medicine. Current literature mainly focuses on deriving ITRs from a single source population. We consider the observational data setting when the source population differs from a target population of interest. We assume subject covariates are available from both populations, but treatment and outcome data are only available from the source population. Although adjusting for differences between source and target populations can potentially lead to an improved ITR for the target population, it can substantially increase the variability in ITR estimation. To address this dilemma, we develop a weighting framework that aims to tailor an ITR for a given target population and protect against high variability due to superfluous covariate shift adjustments. Our method seeks covariate balance over a nonparametric function class characterized by a reproducing kernel Hilbert space and can improve many ITR learning methods that rely on weights. We show that the proposed method encompasses importance weights and the so-called overlap weights as two extreme cases, allowing for a better bias-variance trade-off in between. Numerical examples demonstrate that the use of our weighting method can greatly improve ITR estimation for the target population compared with other weighting methods.

  Access Paper or Ask Questions

Condensation-Net: Memory-Efficient Network Architecture with Cross-Channel Pooling Layers and Virtual Feature Maps

Apr 29, 2021
Tse-Wei Chen, Motoki Yoshinaga, Hongxing Gao, Wei Tao, Dongchao Wen, Junjie Liu, Kinya Osa, Masami Kato

"Lightweight convolutional neural networks" is an important research topic in the field of embedded vision. To implement image recognition tasks on a resource-limited hardware platform, it is necessary to reduce the memory size and the computational cost. The contribution of this paper is stated as follows. First, we propose an algorithm to process a specific network architecture (Condensation-Net) without increasing the maximum memory storage for feature maps. The architecture for virtual feature maps saves 26.5% of memory bandwidth by calculating the results of cross-channel pooling before storing the feature map into the memory. Second, we show that cross-channel pooling can improve the accuracy of object detection tasks, such as face detection, because it increases the number of filter weights. Compared with Tiny-YOLOv2, the improvement of accuracy is 2.0% for quantized networks and 1.5% for full-precision networks when the false-positive rate is 0.1. Last but not the least, the analysis results show that the overhead to support the cross-channel pooling with the proposed hardware architecture is negligible small. The extra memory cost to support Condensation-Net is 0.2% of the total size, and the extra gate count is only 1.0% of the total size.

* 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 
* Camera-ready version for CVPR 2019 workshop (Embedded Vision Workshop) 

  Access Paper or Ask Questions

On the Whitney extension problem for near isometries and beyond

Mar 17, 2021
Steven B. Damelin

This paper is an exposition of work of the author et al. detailing fascinating connections between several mathematical problems which lie on the intersection of several mathematics subjects, namely algebraic-differential geometry, analysis on manifolds, complex-harmonic analysis, data science, partial differential equations, optimization and probability. A significant portion of the work is based on joint research with Charles Fefferman in the papers [39, 40, 41, 42]. The topics of this work include (a) The space of maps of bounded mean oscillation (BMO) in $\mathbb R^D,\, D\geq 2$. (b) The labeled and unlabeled near alignment and Procrustes problem for point sets with certain geometries and for not too thin compact sets both in $\mathbb R^D,\, D\geq 2$. (c) The Whitney near isometry extension problem for point sets with certain geometries and for not too thin compact sets both in $\mathbb R^D,\, D\geq 2$. (d) Partitions and clustering of compact sets and point sets with certain geometries in $\mathbb R^D,\, D\geq 2$ and analysis on certain manifolds in $\mathbb R^D,\, D\geq 2$. Many open problems for future research are given.

  Access Paper or Ask Questions

A Novel Word Sense Disambiguation Approach Using WordNet Knowledge Graph

Jan 08, 2021
Mohannad AlMousa, Rachid Benlamri, Richard Khoury

Various applications in computational linguistics and artificial intelligence rely on high-performing word sense disambiguation techniques to solve challenging tasks such as information retrieval, machine translation, question answering, and document clustering. While text comprehension is intuitive for humans, machines face tremendous challenges in processing and interpreting a human's natural language. This paper presents a novel knowledge-based word sense disambiguation algorithm, namely Sequential Contextual Similarity Matrix Multiplication (SCSMM). The SCSMM algorithm combines semantic similarity, heuristic knowledge, and document context to respectively exploit the merits of local context between consecutive terms, human knowledge about terms, and a document's main topic in disambiguating terms. Unlike other algorithms, the SCSMM algorithm guarantees the capture of the maximum sentence context while maintaining the terms' order within the sentence. The proposed algorithm outperformed all other algorithms when disambiguating nouns on the combined gold standard datasets, while demonstrating comparable results to current state-of-the-art word sense disambiguation systems when dealing with each dataset separately. Furthermore, the paper discusses the impact of granularity level, ambiguity rate, sentence size, and part of speech distribution on the performance of the proposed algorithm.

  Access Paper or Ask Questions

A CRF-based Framework for Tracklet Inactivation in Online Multi-Object Tracking

Nov 30, 2020
Tianze Gao, Huihui Pan, Zidong Wang, Huijun Gao

Online multi-object tracking (MOT) is an active research topic in the domain of computer vision. In this paper, a CRF-based framework is put forward to tackle the tracklet inactivation issues in online MOT problems. We apply the proposed framework to one of the state-of-the-art online MOT trackers, Tracktor++. The baseline algorithm for online MOT has the drawback of simple strategy on tracklet inactivation, which relies merely on tracking hypotheses' classification scores partitioned by using a fixed threshold. To overcome such a drawback, a discrete conditional random field (CRF) is developed to exploit the intra-frame relationship between tracking hypotheses. Separate sets of feature functions are designed for the unary and binary terms in the CRF so as to cope with various challenges in practical situations. The hypothesis filtering and dummy nodes techniques are employed to handle the problem of varying CRF nodes in the MOT context. In this paper, the inference of CRF is achieved by using the loopy belief propagation algorithm, and the parameters of the CRF are determined by utilizing the maximum likelihood estimation method. Experimental results demonstrate that the developed tracker with our CRF-based framework outperforms the baseline on the MOT16 and MOT17 datasets. The extensibility of the proposed method is further validated by an extensive experiment.

* 13 pages, 9 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible 

  Access Paper or Ask Questions

SafeML: Safety Monitoring of Machine Learning Classifiers through Statistical Difference Measure

May 27, 2020
Koorosh Aslansefat, Ioannis Sorokos, Declan Whiting, Ramin Tavakoli Kolagari, Yiannis Papadopoulos

Ensuring safety and explainability of machine learning (ML) is a topic of increasing relevance as data-driven applications venture into safety-critical application domains, traditionally committed to high safety standards that are not satisfied with an exclusive testing approach of otherwise inaccessible black-box systems. Especially the interaction between safety and security is a central challenge, as security violations can lead to compromised safety. The contribution of this paper to addressing both safety and security within a single concept of protection applicable during the operation of ML systems is active monitoring of the behaviour and the operational context of the data-driven system based on distance measures of the Empirical Cumulative Distribution Function (ECDF). We investigate abstract datasets (XOR, Spiral, Circle) and current security-specific datasets for intrusion detection (CICIDS2017) of simulated network traffic, using distributional shift detection measures including the Kolmogorov-Smirnov, Kuiper, Anderson-Darling, Wasserstein and mixed Wasserstein-Anderson-Darling measures. Our preliminary findings indicate that the approach can provide a basis for detecting whether the application context of an ML component is valid in the safety-security. Our preliminary code and results are available at

  Access Paper or Ask Questions

One-Shot Unsupervised Cross-Domain Detection

May 23, 2020
Antonio D'Innocente, Francesco Cappio Borlino, Silvia Bucci, Barbara Caputo, Tatiana Tommasi

Despite impressive progress in object detection over the last years, it is still an open challenge to reliably detect objects across visual domains. Although the topic has attracted attention recently, current approaches all rely on the ability to access a sizable amount of target data for use at training time. This is a heavy assumption, as often it is not possible to anticipate the domain where a detector will be used, nor to access it in advance for data acquisition. Consider for instance the task of monitoring image feeds from social media: as every image is created and uploaded by a different user it belongs to a different target domain that is impossible to foresee during training. This paper addresses this setting, presenting an object detection algorithm able to perform unsupervised adaption across domains by using only one target sample, seen at test time. We achieve this by introducing a multi-task architecture that one-shot adapts to any incoming sample by iteratively solving a self-supervised task on it. We further enhance this auxiliary adaptation with cross-task pseudo-labeling. A thorough benchmark analysis against the most recent cross-domain detection methods and a detailed ablation study show the advantage of our method, which sets the state-of-the-art in the defined one-shot scenario.

  Access Paper or Ask Questions

Unsupervised Cross-Modal Audio Representation Learning from Unstructured Multilingual Text

Mar 27, 2020
Alexander Schindler, Sergiu Gordea, Peter Knees

We present an approach to unsupervised audio representation learning. Based on a triplet neural network architecture, we harnesses semantically related cross-modal information to estimate audio track-relatedness. By applying Latent Semantic Indexing (LSI) we embed corresponding textual information into a latent vector space from which we derive track relatedness for online triplet selection. This LSI topic modelling facilitates fine-grained selection of similar and dissimilar audio-track pairs to learn the audio representation using a Convolution Recurrent Neural Network (CRNN). By this we directly project the semantic context of the unstructured text modality onto the learned representation space of the audio modality without deriving structured ground-truth annotations from it. We evaluate our approach on the Europeana Sounds collection and show how to improve search in digital audio libraries by harnessing the multilingual meta-data provided by numerous European digital libraries. We show that our approach is invariant to the variety of annotation styles as well as to the different languages of this collection. The learned representations perform comparable to the baseline of handcrafted features, respectively exceeding this baseline in similarity retrieval precision at higher cut-offs with only 15\% of the baseline's feature vector length.

* In Proceedings of the 35th ACM/SIGAPP Symposium On Applied Computing (SAC2020), March 30-April 3, 2020, Brno, Czech Republic 
* This is the long version of our SAC2020 poster presentation 

  Access Paper or Ask Questions