📚 Pretraining

🧠 NeurIPS2026 · 1 paper notes

📌 Same area in other venues: 🎞️ ECCV2026 (15) · 📷 CVPR2026 (5) · 🔬 ICLR2026 (79) · 💬 ACL2026 (12) · 🧪 ICML2026 (27) · 🤖 AAAI2026 (9)

Spectral-Sphere-Constrained Hyper-Connections

s²HC replaces nonnegative doubly stochastic constraints on multi-stream residual matrices with a mean-preserving spectral norm sphere, using dynamic rotations and bounded scaling in the zero-sum subspace to enable non-degenerate mixing and achieving average accuracy of 47.1, 50.2, and 50.7 across eight benchmarks on three language models pretrained from scratch.