The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems

Описание: The speaker motivates embedding compression challenges as embedding tables grow with entity cardinality and dimensionality. They survey alternatives such as low-dimensional approximations, quantization, and offloading, noting quality, portability, and latency issues. CompressSAE uses a linear encoder with top-k activation and a bias-free linear decoder with row normalization to produce sparse embeddings that preserve cosine similarity. Retrieval can use standard sparse dot product or a kernel trick to simulate original-space similarity. Training requires only existing embeddings, not raw data, and converges quickly. Experiments show stronger accuracy–compression trade-offs than Matryoshka, faster inference, and competitive online results. Sparsity is controlled via the hyperparameter k.