Radial Neighborhood Smoothing Recommender System

Conference: NeurIPS 2025 arXiv: 2507.09952 Code: To be confirmed Area: Recommender Systems / Matrix Completion Keywords: Recommender System, Collaborative Filtering, SVD, Kernel Smoothing, Cold-start

TL;DR

This paper proposes the Radial Neighborhood Estimator (RNE), which approximates latent space distances using the row/column L2 norms of the observed matrix, constructs radial neighborhoods encompassing both overlapping and partially overlapping user–item pairs, and applies local kernel regression for smoothed imputation. RNE outperforms conventional collaborative filtering and matrix factorization methods in both theoretical guarantees and empirical evaluations, while naturally alleviating the cold-start problem.

Background & Motivation

Background: Two dominant paradigms in recommender systems are matrix factorization (SVD/nuclear norm) and neighborhood methods (collaborative filtering). The former captures global latent factors but overlooks local patterns, while the latter leverages similar neighbors but is constrained by the number of co-rated items.

Limitations of Prior Work: (a) Collaborative filtering requires a sufficient number of co-rated items between two users to compute similarity, limiting its effectiveness on sparse data; (b) matrix factorization captures only global features, ignoring local neighborhood information and lacking interpretability; (c) new users/items with no interaction history suffer from the cold-start problem.

Key Challenge: Distances in the latent space are not directly observable. How can latent space distances be accurately estimated from a noisy, heavily incomplete observed rating matrix?

Goal: Bridge the gap between latent factor distances and observed matrix distances, constructing more effective neighborhood sets for rating prediction.

Key Insight: Based on SVD decomposition, the paper proves that row/column L2 norms are equivalent to latent factor distances (Proposition 1), and corrects for noise-induced bias using empirical variance (Proposition 2), enabling estimation of latent space distances solely from observed values.

Core Idea: Approximate latent space distances using noise-corrected row/column L2 distances of the observed matrix, construct radial neighborhoods, and perform predictions via kernel smoothing.

Method

Overall Architecture

Input: Sparse observed rating matrix \(\mathbf{A}\) (\(n\) users × \(m\) items) Output: Prediction for unobserved entry \(z_{u,i}\)

Key Designs

1. Latent Space Distance Estimation:

   • Function: Estimate inter-user/inter-item latent space distances from the observed matrix.
   • Mechanism: \(d_{u,v}^2 = \frac{1}{|\mathcal{I}_{uv}|}\sum_{i \in \mathcal{I}_{uv}}(a_{u,i} - a_{v,i})^2\), approximating latent factor distances via differences in co-observed ratings.
   • Proposition 1: \(\|\mathbf{Z}_{(u)} - \mathbf{Z}_{(v)}\|_2 = \|\mathbf{x}_u - \mathbf{x}_v\|_2\)
   • Proposition 2: Empirical variance \(\hat{\sigma}^2\) is used to correct for the non-centrality introduced by noise.
   • Design Motivation: Requires no lower bound on the number of co-rated items and no external covariates.

2. Radial Neighborhood Construction:

   • Function: Construct neighborhoods encompassing both direct (overlapping) and indirect (partially overlapping) user–item pairs.
   • Mechanism: For a target pair \((u, i)\), the method incorporates not only similar users who have rated item \(i\) (first-order neighbors) but also users similar to \(u\) who have not rated \(i\) but are indirectly linked through similar items (second-order neighbors), forming the radial neighborhood set \(\mathcal{N}_{u,i}\).
   • Design Motivation: Incorporating more observed ratings improves prediction robustness, particularly for cold-start users/items.

3. Kernel Smoothing Prediction:

   • Function: Perform weighted-average prediction via kernel regression within the radial neighborhood.
   • Mechanism: \(\hat{z}_{u,i} = \frac{\sum_{(v,j) \in \mathcal{N}_{u,i}} K_h(\hat{d}_{u,v}, \hat{d}_{i,j}) \cdot a_{v,j}}{\sum K_h}\), where the kernel function \(K_h\) weights contributions based on estimated distances.
   • Design Motivation: The nonparametric framework offers high flexibility and the ability to model complex structural similarity relationships in the network.

Loss & Training

• Two-step estimation: The first step applies simple matrix factorization to obtain an initial estimate \(\hat{z}^{(1)}\), used to estimate the noise variance \(\hat{\sigma}^2\); the second step applies RNE for refined prediction.
• Theoretical guarantees: Theorem 1 proves the consistency of the distance estimator; asymptotic analysis is provided.

Key Experimental Results

Main Results

Dataset	Metric	RNE	SoftImpute	UserKNN	ItemKNN
MovieLens 100K	RMSE	0.912	0.943	0.968	0.955
MovieLens 1M	RMSE	0.855	0.872	0.901	0.889
Book-Crossing	RMSE	1.521	1.589	1.643	1.601

Ablation Study

Configuration	Key Findings
Without radial neighborhood (first-order only)	Performance degrades; second-order neighborhoods are important.
Without noise correction	Distance estimates are inflated, leading to inaccurate neighborhood construction.
Different kernel functions	Gaussian kernel achieves the best performance.

Key Findings

• RNE demonstrates a clear advantage in sparse settings: performance gains over baselines are larger at higher missing rates.
• Cold-start mitigation: multi-level neighborhoods provide sufficient information for prediction even for new users.
• Theoretical asymptotic analysis is in strong agreement with empirical results.

Highlights & Insights

• Theoretical elegance: A three-step derivation (latent factor distance = true matrix row distance → approximation via observed matrix row distance → consistency under sparse observations) bridges matrix factorization and collaborative filtering.
• The radial neighborhood design extends the traditional "co-rated items" notion of neighbors, allowing indirectly related user–item pairs to participate in prediction.
• The nonparametric kernel smoothing framework is flexible and requires minimal hyperparameter tuning.

Limitations & Future Work

• Strong MCAR assumption: Assumption 1 presupposes data missing completely at random, whereas real-world recommendation data are typically missing not at random.
• Linear low-rank assumption: The SVD model assumes linear latent factors, limiting its capacity to model nonlinear preference relationships.
• Absence of deep learning baselines: No comparison with neural collaborative filtering methods such as NCF or AutoRec.
• Future directions: Integration with deep learning methods; relaxation of the MCAR assumption.

Related Work & Insights

• vs. Matrix Factorization: Matrix factorization captures only global features, whereas RNE additionally exploits local neighborhood information.
• vs. Traditional Collaborative Filtering: Conventional CF requires a sufficient number of co-rated items; RNE relaxes this constraint through distance estimation.
• vs. Deep Recommenders: RNE is a statistical method with strong theoretical guarantees, but may underperform deep models on large-scale, complex datasets.

Rating

• Novelty: ⭐⭐⭐⭐ The theoretical perspective of bridging two major recommendation paradigms via distance estimation is highly original.
• Experimental Thoroughness: ⭐⭐⭐ Multiple real-world datasets are used, but deep learning baselines are absent.
• Writing Quality: ⭐⭐⭐⭐ Mathematically rigorous with clear theoretical derivations.
• Value: ⭐⭐⭐⭐ Provides new theoretical insights for recommender systems.

Related Papers

• [NeurIPS 2025] R²ec: Towards Large Recommender Models with Reasoning
• [NeurIPS 2025] Think before Recommendation: Autonomous Reasoning-enhanced Recommender
• [ICLR 2026] Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systems
• [ICLR 2026] Token-Efficient Item Representation via Images for LLM Recommender Systems
• [NeurIPS 2025] Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning