Information Propagation by Composited Labels in Natural Language Processing

In natural language processing (NLP), labeling on regions of text, such as words, sentences and paragraphs, is a basic task. In this paper, label is defined as map between mention of entity in a region on text and context of entity in a broader region on text containing the mention. This definition naturally introduces linkage of entities induced from inclusion relation of regions, and connected entities form a graph representing information flow defined by map. It also enables calculation of information loss through map using entropy, and entropy lost is regarded as distance between two entities over a path on graph.

Supervising Unsupervised Learning with Evolutionary Algorithm in Deep Neural Network

A method to control results of gradient descent unsupervised learning in a deep neural network by using evolutionary algorithm is proposed. To process crossover of unsupervisedly trained models, the algorithm evaluates pointwise fitness of individual nodes in neural network. Labeled training data is randomly sampled and breeding process selects nodes by calculating degree of their consistency on different sets of sampled data. This method supervises unsupervised training by evolutionary process. We also introduce modified Restricted Boltzmann Machine which contains repulsive force among nodes in a neural network and it contributes to isolate network nodes each other to avoid accidental degeneration of nodes by evolutionary process. These new methods are applied to document classification problem and it results better accuracy than a traditional fully supervised classifier implemented with linear regression algorithm.