Interpretability in Deep Time Series Models Demands Semantic Alignment¶

Conference: ICML 2026
arXiv: 2602.02239
Code: To be confirmed
Area: Time Series / Interpretability
Keywords: Semantic Alignment, Interpretability, Time Series, Concept Bottleneck, Neuro-symbolic

TL;DR¶

This is a position paper—proposing that deep time series models should enforce semantic alignment: ensuring that internal variables and mechanisms correspond to the reasoning processes of domain experts rather than merely explaining internal computations. The core innovation is the definition of persistence constraints for semantic alignment regarding temporal evolution (a challenge unique to time series).

Background & Motivation¶

Background: Deep learning has achieved significant success in time series forecasting, yet the black-box nature of these models limits their application in high-stakes domains such as finance and healthcare. Existing interpretability methods (attention mechanisms, post-hoc explanations, mechanistic interpretability) primarily attempt to explain internal model computations.

Limitations of Prior Work: These methods only address structural opacity (how internal computations are understood) but fail to resolve semantic opacity. For example, a clinician cannot interpret the meaning of a "latent variable activation at timestep 47" because it does not map to a recognized medical concept (e.g., "tachycardia onset").

Key Challenge: Even if a model's predictions are accurate, users cannot meaningfully verify, debug, or intervene in model behavior because the conceptual level at which the model operates does not match the user's reasoning level.

Goal: (1) Formally define semantic alignment in the context of time series; (2) Provide a design blueprint for interpretable time series models; (3) Discuss the properties that underpin trustworthiness and identify new design opportunities.

Key Insight: Inspired by Concept Bottleneck Models (CBM) in computer vision, but recognizing that existing CBM approaches are unsuitable for time series due to their lack of semantic alignment guarantees across temporal evolution.

Core Idea: Extend Concept Bottleneck Models to the temporal domain by decomposing the model into [Concept Encoding → Concept Propagation → Task Decoding] and constraining the propagation mechanism to satisfy domain knowledge constraints.

Method¶

Overall Architecture¶

General deep models adopt an Encoding-Propagation-Decoding template: $$\mathbf{u}_t = \text{Enc}(\mathbf{x}_{\leq t}), \quad \mathbf{z}_{t+1} = \text{Prop}(\mathbf{z}_{\leq t}, \mathbf{u}_t), \quad \hat{\mathbf{y}} = \text{Dec}(\mathbf{z}_{t+1})$$ where $\mathbf{u}_t$ represents the instantaneous representation generated by the encoder, and $\mathbf{z}_t$ represents the dynamic representation produced by the propagation layer.

Key Designs¶

Formalization of Semantic Opacity:
- Function: Distinguishes between structural opacity (explaining internal computations) and semantic opacity (inability to express reasoning via domain concepts).
- Mechanism: Introduces "concepts" (human-interpretable random variables) and "mechanisms" (conditional probability distributions between concepts $P(V_{\text{out}} \mid V_{\text{in}})$), defining semantic alignment as the matching of model representations to domain concepts.
- Design Motivation: Prior work either focuses solely on structural calculations or ignores the degradation of alignment during temporal evolution (where alignment at time $t$ might drift by $t+1$).
Binary Partitioning of Instantaneous and Dynamic Concepts:
- Function: Defines two categories of concepts relevant to the user.
- Mechanism: Instantaneous concepts $C_t^U$ represent a "snapshot" of the system's current state regardless of temporal evolution (e.g., "current temperature exceeds threshold"); Dynamic concepts $C_t^Z$ are concepts whose future values the user wishes to predict, requiring semantic persistence over time (e.g., "thermal stress accumulation"). Formal semantic alignment is defined as $P(U_t = C_t^U \mid \mathbf{x}_{\leq t}) = 1$ and $P(Z_{t+1} = C_{t+1}^Z \mid \mathbf{x}_{\leq t}) = 1$. The second constraint has no analogue in static models and is a unique contribution of this work for time series.
- Design Motivation: Satisfying alignment at time $t$ without ensuring maintenance at $t+1$ leads to an exponential decay of semantic alignment, ultimately rendering the model untrustworthy.
Mechanism Alignment as a Constraint Satisfaction Problem:
- Function: Goes beyond concept alignment to require that the way models represent relationships between concepts aligns with user understanding.
- Mechanism: Defines mechanism alignment as $P(V_{\text{out}} \mid V_{\text{in}}) \in \mathcal{M}^{(h)}_{V_{\text{out}} \mid V_{\text{in}}}$, where $\mathcal{M}^{(h)}$ is a family of conditional probability distributions acceptable to the user (which can be specified as monotonic functions, linear relationships, or physical constraints).
- Design Motivation: Empowers users with control over model reasoning steps, supporting formal verification and human-computer interaction.

Key Experimental Results¶

Main Results¶

Interpretability Paradigm	Instantaneous Concept Alignment	Dynamic Concept Alignment	Mechanism Alignment
Input Importance / Proxy Models / Post-hoc	✗	✗	✗
Attention Mechanisms	✗	✗	✗
Koopman Linearization	✗	~	~
Symbolic Regression	~	~	✓
Mechanistic Interpretability	✗	✗	✗
Prototype Methods	~	✗	✗
Physics-Informed Constraints	~	~	✓
Ours (Semantic Alignment)	✓	✓	✓

Ablation Study¶

Design Option	Key Property	Description
Instantaneous Alignment Only	Incomplete	Cannot guarantee semantic stability during temporal evolution
Adding Dynamic Alignment	Necessary	Prevents the exponential decay of semantic drift
3-term Loss (Task + Concept + Propagation) vs. 2-term	Critical	Removing propagation loss leads to concept alignment failure in long-term forecasting

Key Findings¶

Necessity of Dynamic Alignment: If the second alignment constraint is ignored, even if concept predictions at each individual timestep are accurate, the model will deviate from user-understood concept trajectories after multi-step propagation—a problem unique to time series.
Relationship with Static CBM: The framework is directly compatible with existing advances in Concept Bottleneck Models (probabilistic concepts, concept embeddings, etc.) but adds temporal dimension constraints.
Mitigating the Accuracy-Interpretability Trade-off: Through residual paths, concept embeddings, or unsupervised concepts, semantically aligned models can maintain accuracy comparable to black-box models.

Highlights & Insights¶

Innovation in Conceptual Framework: Repositions interpretability from "explaining internal calculations" to "ensuring concepts and mechanisms align with user mental models"—a perspective shift that is enlightening for the field.
Unique Challenges of Time Series: Unlike static models, time series models must maintain semantic alignment across multiple timesteps; post-hoc explanations or attention visualizations cannot solve this—alignment must be enforced at the design level.
Transferable Design Principles: The blueprint is applicable to various time series tasks (forecasting, classification, generation) and points toward the integration of neuro-symbolic methods, formal verification, and time series analysis.
Rational Critique of Existing Methods: Systematically demonstrates through Table 1 that existing methods like mechanistic interpretability or linearization either lack concept alignment, mechanism alignment, or ignore dynamic alignment—making a persuasive case.

Limitations & Future Work¶

Annotation Bottleneck: Achieving semantic alignment requires substantial concept-level annotation; the paper acknowledges this but suggests alternatives (LLM labeling, concept discovery algorithms, formal constraints).
Lack of Complete Formal Theory: The paper focuses on definitions and blueprints but does not provide a complete theory of interpretability (e.g., quantifying the degree of alignment or formal verification algorithms).
Absence of Practical Systems: As a pure position paper, there is no specific system implementation or case study to validate the feasibility of the blueprint.
Trade-offs in Mechanism Alignment: Forcing mechanism alignment via physical constraints or modular composition is discussed, but the impact on accuracy and the balance between satisfying constraints and maintaining expressivity are not explored in depth.

vs. Traditional Interpretability (LIME, SHAP): These methods explain single predictions but do not build testable, intervenable semantic structures; this paper emphasizes that post-hoc explanations cannot guarantee alignment.
vs. Neuro-symbolic Methods: These attempt to combine symbolic reasoning, but most work is in static or simple dynamic settings; this paper extends them to a complete time series framework.
vs. Koopman / Linearized Dynamics: These methods study learning within spatially constrained models but are not necessarily aligned with user concepts; this paper adds concept-level constraints.
vs. Concept Bottleneck Models (CBM): Extant CBM literature primarily targets static classification; the primary contribution of this paper is the formalization of semantic alignment for temporal propagation layers.

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First to systematically formalize semantic alignment in time series, pushing CBM from static to dynamic and introducing persistence constraints for dynamic alignment.
Experimental Thoroughness: ⭐⭐⭐ As a position paper, it lacks experimental data but supports its views through comparative tables, counter-arguments, and design blueprints; a prototype system would have been more persuasive.
Writing Quality: ⭐⭐⭐⭐⭐ Clear logic, consistent notation, and strong motivation; the running example (industrial equipment fault diagnosis) throughout the text aids understanding.
Value: ⭐⭐⭐⭐⭐ Significant guiding importance for the time series interpretability community; formalizes long-overlooked issues, provides an operational blueprint, and identifies at least five new research directions.