A Neuromorphic Model of Learning Meaningful Sequences with Long-Term Memory

Learning meaningful sentences is different from learning a random set of words. When humans understand the meaning, the learning occurs relatively quickly. What mechanisms enable this to happen? In this paper, we examine the learning of novel sequences in familiar situations. We embed the Small World of Words (SWOW-EN), a Word Association Norms (WAN) dataset, in a spiking neural network based on the Hierarchical Temporal Memory (HTM) model to simulate long-term memory. Results show that in the presence of SWOW-EN, there is a clear difference in speed between the learning of meaningful sentences and random noise. For example, short poems are learned much faster than sequences of random words. In addition, the system initialized with SWOW-EN weights shows greater tolerance to noise.