Parallel Collaborative Filtering for the Netflix Prize (algorithm review) | AISC Foundational

Описание: Toronto Deep Learning Series, 17-Jan-2019
https://tdls.a-i.science/events/2019-...

Paper: https://endymecy.gitbooks.io/spark-ml...

Discussion Panel: Preston Engstrom, Serena McDonnell, Omar Nada

Host: IEEE student chapter at Ryerson University

LARGE-SCALE PARALLEL COLLABORATIVE FILTERING FOR THE NETFLIX PRIZE

Many recommendation systems suggest items to users by
utilizing the techniques of collaborative filtering (CF) based on historical records of items that the users have viewed, purchased, or rated. Two major problems that most CF approaches have to resolve are scalability and sparseness of the user profiles. In this paper, we describe Alternating-Least-Squares with Weighted-λ-Regularization (ALS-WR), a parallel algorithm that we designed for the Netflix Prize, a large-scale collaborative filtering challenge. We use parallel Matlab on a Linux cluster as the experimental platform. We show empirically that the performance of ALS-WR monotonically increases with both the number of features and the number of ALS iterations. Our ALS-WR applied to the Netflix dataset with 1000 hidden features obtained a RMSE score of 0.8985, which is one of the best results based on a pure method. Combined with the parallel version of other known methods, we achieved a performance improvement of 5.91% over Netflix’s own CineMatch recommendation system. Our method is simple and scales well to very large datasets.