An optimal pairwise merge algorithm improves the quality and consistency of nonnegative matrix factorization

Non-negative matrix factorization (NMF) is a key technique for feature extraction and widely used in source separation. However, existing algorithms may converge to poor local minima, or to one of several minima with similar objective value but differing feature parametrizations. Additionally, the performance of NMF greatly depends on the number of components, but choosing the optimal count remains a challenge. Here we show that some of these weaknesses may be mitigated by performing NMF in a higher-dimensional feature space and then iteratively combining components with an analytically-solvable pairwise merge strategy. Experimental results demonstrate our method helps NMF achieve better local optima and greater consistency of the solutions. Iterative merging also provides an efficient and informative framework for choosing the number of components. Surprisingly, despite these extra steps, our approach often improves computational performance by reducing the occurrence of ``convergence stalling'' near saddle points. This can be recommended as a preferred approach for most applications of NMF.