Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

"Information": models, code, and papers

Information Bottlenecks, Causal States, and Statistical Relevance Bases: How to Represent Relevant Information in Memoryless Transduction

Jun 16, 2000
Cosma Rohilla Shalizi, James P. Crutchfield

Discovering relevant, but possibly hidden, variables is a key step in constructing useful and predictive theories about the natural world. This brief note explains the connections between three approaches to this problem: the recently introduced information-bottleneck method, the computational mechanics approach to inferring optimal models, and Salmon's statistical relevance basis.

* Advances in Complex Systems, vol. 5, pp. 91--95 (2002) 
* 3 pages, no figures, submitted to PRE as a "brief report". Revision: added an acknowledgements section originally omitted by a LaTeX bug 

  Access Paper or Ask Questions

End-to-End Learning of Joint Geometric and Probabilistic Constellation Shaping

Dec 09, 2021
Vahid Aref, Mathieu Chagnon

We present a novel autoencoder-based learning of joint geometric and probabilistic constellation shaping for coded-modulation systems. It can maximize either the mutual information (for symbol-metric decoding) or the generalized mutual information (for bit-metric decoding).

* Will be presented at OFC 2022 (invited talk) 

  Access Paper or Ask Questions

Dienstplanerstellung in Krankenhaeusern mittels genetischer Algorithmen

May 30, 2013
Uwe Aickelin

This thesis investigates the use of problem-specific knowledge to enhance a genetic algorithm approach to multiple-choice optimisation problems. It shows that such information can significantly enhance performance, but that the choice of information and the way it is included are important factors for success.

* Diplomarbeit, in German, Universitaet Mannheim, 1996 

  Access Paper or Ask Questions

Coherence, Belief Expansion and Bayesian Networks

Mar 08, 2000
Luc Bovens, Stephan Hartmann

We construct a probabilistic coherence measure for information sets which determines a partial coherence ordering. This measure is applied in constructing a criterion for expanding our beliefs in the face of new information. A number of idealizations are being made which can be relaxed by an appeal to Bayesian Networks.

* 6 pages, 2 figures, paper presented at the 8th Intl. Workshop on Non-Monotonic Reasoning NMR'2000 (April 9-11), Breckenridge, Colorado 

  Access Paper or Ask Questions

A Short Introduction to Model Selection, Kolmogorov Complexity and Minimum Description Length (MDL)

May 14, 2010
Volker Nannen

The concept of overfitting in model selection is explained and demonstrated with an example. After providing some background information on information theory and Kolmogorov complexity, we provide a short explanation of Minimum Description Length and error minimization. We conclude with a discussion of the typical features of overfitting in model selection.

* 20 pages, Chapter 1 of The Paradox of Overfitting, Master's thesis, Rijksuniversiteit Groningen, 2003 

  Access Paper or Ask Questions

The Choquet integral for the aggregation of interval scales in multicriteria decision making

Apr 10, 2008
Christophe Labreuche, Michel Grabisch

This paper addresses the question of which models fit with information concerning the preferences of the decision maker over each attribute, and his preferences about aggregation of criteria (interacting criteria). We show that the conditions induced by these information plus some intuitive conditions lead to a unique possible aggregation operator: the Choquet integral.

* Fuzzy Sets and Systems 137 (2003) 11-26 

  Access Paper or Ask Questions

Lex2vec: making Explainable Word Embedding via Distant Supervision

Mar 03, 2021
Fabio Celli

In this technical report we propose an algorithm, called Lex2vec, that exploits lexical resources to inject information into word embeddings and name the embedding dimensions by means of distant supervision. We evaluate the optimal parameters to extract a number of informative labels that is readable and has a good coverage for the embedding dimensions.

* 3 pages, 1 figure, 1 table 

  Access Paper or Ask Questions

On Implementing Usual Values

Mar 27, 2013
Ronald R. Yager

In many cases commonsense knowledge consists of knowledge of what is usual. In this paper we develop a system for reasoning with usual information. This system is based upon the fact that these pieces of commonsense information involve both a probabilistic aspect and a granular aspect. We implement this system with the aid of possibility-probability granules.

* Appears in Proceedings of the Second Conference on Uncertainty in Artificial Intelligence (UAI1986) 

  Access Paper or Ask Questions

A Short Survey of Biomedical Relation Extraction Techniques

Jul 25, 2017
Elham Shahab

Biomedical information is growing rapidly in the recent years and retrieving useful data through information extraction system is getting more attention. In the current research, we focus on different aspects of relation extraction techniques in biomedical domain and briefly describe the state-of-the-art for relation extraction between a variety of biological elements.

* updated keywords and reference format 

  Access Paper or Ask Questions

Bounded Planning in Passive POMDPs

Jun 27, 2012
Roy Fox, Naftali Tishby

In Passive POMDPs actions do not affect the world state, but still incur costs. When the agent is bounded by information-processing constraints, it can only keep an approximation of the belief. We present a variational principle for the problem of maintaining the information which is most useful for minimizing the cost, and introduce an efficient and simple algorithm for finding an optimum.

* Appears in Proceedings of the 29th International Conference on Machine Learning (ICML 2012) 

  Access Paper or Ask Questions