An optimal pairwise merge algorithm improves the quality and consistency of nonnegative matrix factorization

Non-negative matrix factorization (NMF) is a key technique for feature extraction and widely used in source separation. However, existing algorithms may converge to poor local minima, or to one of several minima with similar objective value but differing feature parametrizations. Additionally, the performance of NMF greatly depends on the number of components, but choosing the optimal count remains a challenge. Here we show that some of these weaknesses may be mitigated by performing NMF in a higher-dimensional feature space and then iteratively combining components with an analytically-solvable pairwise merge strategy. Experimental results demonstrate our method helps NMF achieve better local optima and greater consistency of the solutions. Iterative merging also provides an efficient and informative framework for choosing the number of components. Surprisingly, despite these extra steps, our approach often improves computational performance by reducing the occurrence of ``convergence stalling'' near saddle points. This can be recommended as a preferred approach for most applications of NMF.

GSVD-NMF: Recovering Missing Features in Non-negative Matrix Factorization

Non-negative matrix factorization (NMF) is an important tool in signal processing and widely used to separate mixed sources into their components. However, NMF is NP-hard and thus may fail to discover the ideal factorization; moreover, the number of components may not be known in advance and thus features may be missed or incompletely separated. To recover missing components from under-complete NMF, we introduce GSVD-NMF, which proposes new components based on the generalized singular value decomposition (GSVD) between preliminary NMF results and the SVD of the original matrix. Simulation and experimental results demonstrate that GSVD-NMF often recovers missing features from under-complete NMF and helps NMF achieve better local optima.