Invented for the Netflix competition, probabilistic matrix factorization (PMF) was demonstrated as a powerful model for collaborative filtering and has since been applied to a wide spectrum of practical machine learning problems. Attracting great research interest, the mathematics of PMF is now understood to fundamentally relate to the classical model of Principle Component Analysis (PCA). Moreover, recent research has extended PMF along two directions. One direction is extending the notion of "matrices" to multidimensional arrays, i.e., tensors, which has led to the synthesis of the Probabilistic Tensor Factorization (PTF) model. The other direction is allowing the matrices involved in PMF to evolve over time, which led to the algorithm of timeSVD++, capturing the temporal dynamics of collaborative filtering.
This talk contains a review of PMF, PCA, PTF, timeSVD++ and their inherent inter-connections. A convenient diagrammatic notation, normal factor graphs (developed in my recent work with Ali Al-Bashabsheh) will be used for pedagogical purpose. It appears that this notation greatly simplifies the involved mathematics, particularly that with tensors.
One point I will like to make in this review is that although timeSVD++ is an algorithm that builds upon PMF, it is primarily heuristic in nature and its formulation no longer carries any probabilistic interpretation. Recognizing this limitation, I then present a very recent development of the Markov Factorization of Matrix Process (MFMP) model in my joint work with Richong Zhang. I will show that MFMP is a natural extension of the PMF to matrix processes; it is capable of capturing temporal dynamics and has a well-principled probabilistic formulation. I will also demonstrate experimentally that a very basic form of MFMP already outperforms the timeSVD++ algorithm equipped with sophisticated heuristics.