Multiplicative update for NMF is bad (Under construction)

See my boss's work “Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization”, 2011.

Without acceleration, MU is slow. With acceleration, A-MU is still, less competitive than A-HALS.

But people still use MU for NMF. This is super unhealthy.

Why people still use MU?

Why MU is bad

  • The concept fo step size contraction

  • Why gradient descent (without being forced to satisfy the non-negativity constraint) plus projection is better

  • Theoretical convergence proof of projected gradient descent (with convergence rate) \ see Bolte().

How about KullbackÔÇôLeibler divergence ?

See Felipe Yanez and Francis Bach, “Primal-Dual Algorithms for Non-negative Matrix Factorization with the Kullback-Leibler Divergence”