An Incidence Geometry approach to Dictionary Learning

We study the Dictionary Learning (aka Sparse Coding) problem of obtaining a sparse representation of data points, by learning \emph{dictionary vectors} upon which the data points can be written as sparse linear combinations. We view this problem from a geometry perspective as the spanning set of a subspace arrangement, and focus on understanding the case when the underlying hypergraph of the subspace arrangement is specified. For this Fitted Dictionary Learning problem, we completely characterize the combinatorics of the associated subspace arrangements (i.e.\ their underlying hypergraphs). Specifically, a combinatorial rigidity-type theorem is proven for a type of geometric incidence system. The theorem characterizes the hypergraphs of subspace arrangements that generically yield (a) at least one dictionary (b) a locally unique dictionary (i.e.\ at most a finite number of isolated dictionaries) of the specified size. We are unaware of prior application of combinatorial rigidity techniques in the setting of Dictionary Learning, or even in machine learning. We also provide a systematic classification of problems related to Dictionary Learning together with various algorithms, their assumptions and performance.

Learning Hierarchical Sparse Representations using Iterative Dictionary Learning and Dimension Reduction

This paper introduces an elemental building block which combines Dictionary Learning and Dimension Reduction (DRDL). We show how this foundational element can be used to iteratively construct a Hierarchical Sparse Representation (HSR) of a sensory stream. We compare our approach to existing models showing the generality of our simple prescription. We then perform preliminary experiments using this framework, illustrating with the example of an object recognition task using standard datasets. This work introduces the very first steps towards an integrated framework for designing and analyzing various computational tasks from learning to attention to action. The ultimate goal is building a mathematically rigorous, integrated theory of intelligence.

Characterizing 1-Dof Henneberg-I graphs with efficient configuration spaces

We define and study exact, efficient representations of realization spaces of a natural class of underconstrained 2D Euclidean Distance Constraint Systems(EDCS) or Frameworks based on 1-dof Henneberg-I graphs. Each representation corresponds to a choice of parameters and yields a different parametrized configuration space. Our notion of efficiency is based on the algebraic complexities of sampling the configuration space and of obtaining a realization from the sample (parametrized) configuration. Significantly, we give purely combinatorial characterizations that capture (i) the class of graphs that have efficient configuration spaces and (ii) the possible choices of representation parameters that yield efficient configuration spaces for a given graph. Our results automatically yield an efficient algorithm for sampling realizations, without missing extreme or boundary realizations. In addition, our results formally show that our definition of efficient configuration space is robust and that our characterizations are tight. We choose the class of 1-dof Henneberg-I graphs in order to take the next step in a systematic and graded program of combinatorial characterizations of efficient configuration spaces. In particular, the results presented here are the first characterizations that go beyond graphs that have connected and convex configuration spaces.