Skip to content

HN-MVTS: HyperNetwork-based Multivariate Time Series Forecasting

Conference: AAAI 2026 arXiv: 2511.08340 Code: github.com/av-savchenko/HN-MVTS Area: Time Series Keywords: Multivariate time series forecasting, HyperNetwork, channel dependence, channel independence, plug-and-play

TL;DR

This paper proposes HN-MVTS, which employs a HyperNetwork to generate channel-specific weights for the final prediction layer, striking a balance between channel-independent (CI) and channel-dependent (CD) modeling. As a plug-and-play module, it improves forecasting accuracy of various backbone models including DLinear, PatchTST, and TSMixer without incurring additional inference overhead.

Background & Motivation

Multivariate time series (MVTS) forecasting requires capturing both temporal patterns and inter-channel correlations. Current mainstream approaches fall into two camps:

Channel-Independent (CI) models: e.g., DLinear, PatchTST, which model each channel independently and ignore cross-channel relationships. Their strengths lie in robustness and effective training data volume (via channel reuse), but they forfeit the ability to exploit cross-channel information.

Channel-Dependent (CD) models: e.g., TSMixer, iTransformer, which jointly model all channels. While theoretically more expressive, they are prone to overfitting under limited data and often underperform CI models in practice.

This "CI-CD dilemma" remains a core unresolved challenge in MVTS forecasting. Prior work such as DUET has attempted channel clustering, but still requires manually designed grouping strategies.

Core Motivation: Can one design a method that adaptively interpolates between CI and CD? When two channels are similar, their parameters should be shared (analogous to CD); when dissimilar, they should be modeled independently (analogous to CI)—all without modifying the base model architecture.

Key Insight: A HyperNetwork can generate channel-specific prediction layer weights conditioned on learnable channel embeddings. When two channel embeddings are close, their generated weights are also close, enabling implicit parameter sharing; when embeddings are distant, the channels are modeled independently, recovering CI behavior.

Method

Overall Architecture

HN-MVTS appends a HyperNetwork module on top of any base forecasting model: - Input: a learnable embedding vector \(\mathbf{z}^{(n)} \in \mathbb{R}^d\) for each channel - HyperNetwork output: weights \(\mathbf{W}_K^{(n)} \in \mathbb{R}^{H \times D}\) for the final layer of the base model - Training: the HyperNetwork and base model are jointly optimized; at inference, the HyperNetwork is discarded and the generated weights are frozen into the base model

Key Designs

  1. HyperNetwork weight generation: The core idea is to use the HyperNetwork only to generate the weights of the final prediction layer, rather than the entire network. For the \(n\)-th channel, the final-layer weights are produced by a simple MLP (or even a linear transformation):

    \(\mathbf{W}_K^{(n)} = \mathbf{W}_\phi^{(n)} \cdot \mathbf{z}^{(n)}\)

where \(\mathbf{W}_\phi^{(n)} \in \mathbb{R}^{H \times D \times d}\) are the HyperNetwork parameters. This design introduces only \(N \cdot H \cdot D \cdot d\) additional parameters—far fewer than training separate models per channel.

Design Motivation: Modifying only the final layer represents the optimal trade-off between parameter efficiency and expressive power. Modifying more layers would cause parameter explosion, while the final layer directly determines the prediction output and thus has the greatest impact on performance.

  1. Channel embedding initialization: The embedding matrix \(\mathbf{Z} = [\mathbf{z}^{(1)}, \ldots, \mathbf{z}^{(N)}] \in \mathbb{R}^{N \times d}\) is initialized via PCA projection of the Pearson correlation matrix (rather than random initialization). Specifically, the inter-channel correlation matrix is computed over the training set and reduced to \(d\) dimensions via PCA to form the initial embeddings.

Design Motivation: Correlation coefficients reflect statistical similarity between channels; initializing embeddings accordingly ensures that similar channels start with proximate embeddings, accelerating convergence. Ablations show that random initialization leads to slightly higher MSE.

  1. Adaptive CI-CD interpolation mechanism: A key contribution of HN-MVTS is the automatic CI-CD switching realized through the embedding space. If channels \(j_1\) and \(j_2\) have similar embeddings (\(\mathbf{z}_{j_1} \approx \mathbf{z}_{j_2}\)), their prediction-layer weights are also similar, meaning training data from \(j_1\) exerts greater influence on the weight learning of \(j_2\) (analogous to CD). In the extreme case of identical embeddings, this reduces to a global shared model; with fully distinct embeddings, it recovers CI behavior.

Design Motivation: This soft switching avoids hard-coding CI or CD strategies, allowing the model to adaptively select the optimal inter-channel modeling mode based on the data.

Loss & Training

  • Loss function: Standard MSE loss, identical to the base model
  • Optimizer: Adam, learning rate 0.0001, batch size 64
  • Lookback window: \(T=336\), forecasting horizons \(H \in \{48, 96, 192, 336\}\)
  • Embedding dimension: \(d \leq N\) (not exceeding the number of channels)
  • Inference acceleration: After training, the generated weights \(\mathbf{W}_K^{(n)}\) are directly copied into the base model's final layer and the HyperNetwork is discarded—inference time is therefore identical to the base model

Key Experimental Results

Main Results

Five backbone models augmented with HN-MVTS are evaluated on 8 datasets (ECL, ETTm1, ETTm2, Weather, PEMS03/04/07/08), reporting average MSE across 4 forecasting horizons:

Dataset Backbone Original MSE +HN-MVTS MSE Gain
Weather (H=48) DLinear 0.1369 0.1115 18.6%
ECL (H=48) TSMixer 0.1377 0.1220 11.4%
PEMS08 (H=48) iTransformer 0.0870 0.0799 8.2%
PEMS07 (H=48) PatchTST 0.0992 0.0888 10.5%
PEMS04 (H=336) iTransformer 0.1533 0.1333 13.0%
PEMS07 (H=336) PatchTST 0.1619 0.1415 12.6%
PEMS08 (H=96) iTransformer 0.1113 0.0957 14.0%

HN-MVTS yields statistically significant improvements (Wilcoxon rank-sum test, \(p<0.05\)) on the majority of dataset–model combinations.

Ablation Study

Configuration Key Observation Notes
Pearson init vs. random init Pearson superior Correlation prior accelerates embedding convergence
Training time overhead +5%–25% Lightest for DLinear (~12%), moderate for Transformer-based models
Inference time Unchanged HyperNetwork discarded at inference
Embedding visualization Similar embeddings learned across different models Embeddings reflect data characteristics, not architectural choices

Key Findings

  1. Greatest gains on high-channel datasets: The PEMS series (170–883 channels) shows the most pronounced improvements, validating the value of inter-channel correlation modeling.
  2. Simpler models benefit more: Linear models such as DLinear achieve substantial performance gains with HN-MVTS, sometimes surpassing unaugmented complex models.
  3. More robust at longer horizons: HN-MVTS effectively mitigates accuracy degradation in long-horizon settings (e.g., \(H=336\)).
  4. Marginal gains in some cases: ETTm1/ETTm2 show limited improvement under certain models, likely because their channel count (7) leaves little room for cross-channel modeling benefits.

Highlights & Insights

  • Plug-and-play: No modification to the base model architecture is required; adding a HyperNetwork only at the final layer is compatible with Linear/MLP/CNN/Transformer backbones.
  • Zero inference overhead: The HyperNetwork is used only during training; at inference, weights are frozen into the base model with no additional computation.
  • Theoretical elegance: CI-CD continuous interpolation is naturally realized through distances in the embedding space, without manual channel grouping design.
  • Parameter efficiency: The additional parameter count \(N \cdot H \cdot D \cdot d\) is, under certain configurations, even smaller than the total parameters of deploying separate CI models per channel.

Limitations & Future Work

  1. Only the final layer is modified: Multi-layer hyperparameterization (e.g., applying HyperNetworks to intermediate layers) could enable deeper inter-channel information sharing, but would substantially increase parameter count.
  2. Limited benefit for very few channels: Datasets such as ETT with only 7 channels offer limited headroom for cross-channel modeling gains.
  3. Not applicable to non-neural models: Gradient boosting and statistical models remain widely used but are currently incompatible with HN-MVTS.
  4. Sensitivity to embedding dimension: The selection of \(d\) must be matched to the number of channels; no automatic tuning mechanism is provided.
  • HyperNetwork literature: Works such as HyperGPA and LPCNet apply HyperNetworks to non-stationary time series and adaptive parameter updates, but not to improving MSE of mainstream MVTS forecasting models.
  • CI-CD balance: The core insight of this paper generalizes to other scenarios requiring a trade-off between sharing and independence, such as multi-task learning and federated learning.
  • Channel embeddings: The learned embeddings reflect dataset characteristics rather than model architecture, and can be used for downstream analysis such as channel clustering and anomaly detection.

Rating

  • Novelty: ⭐⭐⭐⭐ — The HyperNetwork idea is not new, but its application cleverly resolves the CI-CD dilemma in MVTS forecasting.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ — 8 datasets × 5 models, with comprehensive ablations, training time analysis, and embedding visualization.
  • Writing Quality: ⭐⭐⭐⭐ — Clear logic, well-motivated design choices, and intuitive illustrations.
  • Value: ⭐⭐⭐⭐⭐ — Plug-and-play with zero inference overhead; extremely high practical utility.