When is non-negative matrix decomposition unique? Scott Rickard
Andrzej Cichocki
Complex & Adaptive Systems Laboratory University College Dublin Ireland
[email protected]
Laboratory for Advanced Brain Signal Processing Brain Science Institute RIKEN Japan
[email protected]
Abstract— In this paper, we discuss why non-negative matrix factorization (NMF) potentially works for zero-grounded nonnegative components and why it fails when the components are not zero-grounded. We show the demixing process is not uniquely defined (up to the usual permutation/scaling ambiguity) when the original matrices are not zero-grounded. If fact, zerogroundedness alone is not enough. The key observation is that if each component has at least one point for which it is the only active component, the solution is unique. When the non-negative matrices are not zero-grounded, no such point exists and the solution space contains demixtures which are linear combinations of the original components. Thus, the NMF problem has a unique solution for matrices with disjoint components, a condition we call Subset Monomial Disjoint (SMD). The SMD condition is sufficient, but not necessary for NMF to have a unique decomposition, whereas the zero-grounded condition is necessary, but not sufficient.
I. I NTRODUCTION The mixing model that NMF address can be expressed in matrix form, Y = AX + V (1) where A is m-by-r and X is r-by-T and thus Y and V are m-by-T matrices. One interpretation of this model is that the m mixtures which make up the rows of Y are the r original sources stored in the rows of X mixed via mixing matrix A with spatial signatures stored in the columns of A. Another interpretation is that the ‘mixtures’ Y are superpositions of r source signature components in the columns of A stimulated by activation functions in the rows of X. We will refer to the columns of A as the signature component vectors (and to A as the signature matrix) and the rows of X as the stimulus component vectors (and to X as the stimulus matrix). Typically, T >> m ≥ r, although we do not use this assumption in this work. We impose only the weak constraints that m ≥ r and T ≥ r. In this work, we will focus of the ambiguities on the exact decomposition of Y into A and X ignoring the presence of noise V , so we set V = 0. In general, the goal in source separation problems is, given Y , to determine A and/or X. This problem as stated is illdefined, however. Using any r-by-r invertible matrix B, we can state the mixing process as Y = ABB −1 X
(2)
where the signature matrix is now AB and the stimulus matrix is now B −1 X. For every invertible B, we have another
978-1-4244-2247-0/08/$25.00 ©2008 IEEE.
potential factorization. Thus, we must assume some additional conditions on the signature matrix and/or the stimulus matrix in order for a unique solution to be specified. One common assumption is that the stimulus component vectors are independent, an assumption which forms the basis for the field of Independent Component Analysis. An alternate assumption is that the stimulus component vectors are never simultaneously active, the assumption which forms the basis for the DUET blind source separation technique and related methods [1]. Yet another possible assumption is that the signature matrix and stimulus matrix contain only nonnegative entries, an assumption which forms the basis for the field of non-negative matrix factorization (NMF). It is this last case that is the topic of this paper. Note that when B = P D, a permutation multiplied by a diagonal matrix, the elements of B −1 are 1/bji bji 6= 0 −1 bij = (3) 0 otherwise and B −1 is also a permutation times a diagonal matrix. The assumption that A and X are non-negative is not sufficient to eliminate this scaling/permutation degree of freedom as we can choose an arbitrary P and non-negative D which permutes and scales the columns of A and, correspondingly, compensates by permuting and scaling the rows of X. This means that the demixing problem can be solved only up to a arbitrary scaling and permutation of the component vectors. [2] and [3] note that non-negative ICA problem, which is closely related to NMF, has difficulties for sources which are non-zero grounded. In this paper we show that zerogroundedness is not enough. Each signature and stimulus component must have at least one point where it is the only active component in order for NMF to have a unique solution. [4] note the ‘separate support’ result that is the main result of this paper, although the approach taken here is different and emphasizes this disjoint support aspect as the crucial requirement. Our goal is to determine what conditions on A and X result in the demixing problem having a unique solution ignoring the unavoidable scaling/permutation freedom. Stated explicitly: Problem 1 (NMF uniqueness): Given Y ≥ 0, what are the necessary and sufficient conditions on A ≥ 0 and X ≥ 0 such that the only B for which Y = ABB −1 X, AB ≥ 0, and B −1 X ≥ 0 are of the form B = P D for arbitrary
1091
permutation matrix P and arbitrary non-negative non-singular diagonal matrix D. In this paper, we discuss why non-negative matrix factorization (NMF) potentially works for zero-grounded non-negative components and why it fails when the components are not zero-grounded. We show the demixing process is not uniquely defined (up to the usual permutation/scaling ambiguity) when the original matrices are not zero-grounded. If fact, zerogroundedness alone is not enough. The key observation is that if each component has at least one point for which it is the only active component, the solution is unique. When the non-negative matrices are not zero-grounded, no such point exists and the solution space contains demixtures which are linear combinations of the original components. Thus, the NMF problem has a unique solution for matrices with disjoint components, a condition we call Subset Monomial Disjoint (SMD). The SMD condition is sufficient, but not necessary for NMF to have a unique decomposition, whereas the zerogrounded condition is necessary, but not sufficient. R EFERENCES [1] O. Yilmaz and S. Rickard, “Blind separation of speech mixtures via timefrequency masking,” IEEE Transactions on Signal Processing, vol. 52, no. 7, pp. 1830–1847, July 2004. [2] M. D. Plumbley, “Conditions for nonnegative independent component analysis,” IEEE Signal Processing Letter, vol. 9, no. 6, pp. 177–180, June 2002. [3] C.-H. Zheng, D.-S. Huang, Z.-L. Sun, M. Lyu, and T.-M. Lok, “Nonnegative independent component analysis based on minimizing mutual information technique,” Neurocomputing, vol. 69, no. 7–9, pp. 878–883, March 2006. [4] D. Donoho and V. Stodden, “When does non-negative matrix factorization give a correct decomposition into parts?” in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Sch¨olkopf, Eds. Cambridge, MA: MIT Press, 2004.
1092
