A High-Level Model of Neocortical Feedback Based on an Event Window Segmentation Algorithm

The author previously presented an event window segmentation (EWS) algorithm [5] that uses purely statistical methods to learn to recognize recurring patterns in an input stream of events. In the following discussion, the EWS algorithm is first extended to make predictions about future events. Next, this extended algorithm is used to construct a high-level, simplified model of a neocortical hierarchy. An event stream enters at the bottom of the hierarchy, and drives processing activity upward in the hierarchy. Successively higher regions in the hierarchy learn to recognize successively deeper levels of patterns in these events as they propagate from the bottom of the hierarchy. The lower levels in the hierarchy use the predictions from the levels above to strengthen their own predictions. A C++ source code listing of the model implementation and test program is included as an appendix.

A statistical learning algorithm for word segmentation

In natural speech, the speaker does not pause between words, yet a human listener somehow perceives this continuous stream of phonemes as a series of distinct words. The detection of boundaries between spoken words is an instance of a general capability of the human neocortex to remember and to recognize recurring sequences. This paper describes a computer algorithm that is designed to solve the problem of locating word boundaries in blocks of English text from which the spaces have been removed. This problem avoids the complexities of speech processing but requires similar capabilities for detecting recurring sequences. The algorithm relies entirely on statistical relationships between letters in the input stream to infer the locations of word boundaries. A Viterbi trellis is used to simultaneously evaluate a set of hypothetical segmentations of a block of adjacent words. This technique improves accuracy but incurs a small latency between the arrival of letters in the input stream and the sending of words to the output stream. The source code for a C++ version of this algorithm is presented in an appendix.