GenePheno: Interpretable Gene Knockout-Induced Phenotype Abnormality Prediction Framework

Conference: AAAI 2026 Oral
arXiv: 2511.09512
Code: None
Area: Bioinformatics / Multi-label Classification Keywords: Gene sequence, phenotype abnormality prediction, contrastive learning, exclusivity regularization, gene ontology bottleneck layer

TL;DR

This paper proposes GenePheno, the first interpretable multi-label prediction framework for end-to-end prediction of gene knockout-induced phenotype abnormalities directly from gene sequences. The framework captures inter-phenotype correlations via contrastive multi-label learning, enforces biological consistency through exclusivity regularization, and provides interpretability via a Gene Ontology (GO) bottleneck layer. GenePheno achieves state-of-the-art gene-centric \(F_{\max}\) and phenotype-centric AUC across four datasets.

Background & Motivation

Understanding how gene sequences shape phenotypes is a fundamental challenge in biology. Existing approaches fall into two categories: (1) variant effect prediction methods that predict the impact of specific variants on a limited set of phenotypes from gene sequences; and (2) large-scale phenotype abnormality prediction methods that rely on curated information such as protein–protein interaction (PPI) networks or Gene Ontology (GO) annotations as input, suffering from poor scalability and inapplicability to newly discovered or sparsely annotated genes.

Both categories share common limitations: (a) most methods formulate multi-label prediction as independent binary classification tasks, ignoring inherent inter-phenotype correlations (e.g., pleiotropy—where a single gene affects multiple phenotypes); (b) semantic mutual exclusions exist in phenotype ontologies (e.g., hypertonia and hypotonia should not co-occur), yet existing methods cannot guarantee logical consistency; (c) interpretability regarding the intermediate functional mechanisms linking genetic information to phenotype formation is lacking.

• Core Idea: Construct an end-to-end prediction framework from DNA sequences to phenotypes, modeling label correlations via contrastive learning, enforcing biological constraints through exclusivity regularization, and providing interpretable intermediate concepts of phenotype formation mechanisms via a GO bottleneck layer.

Method

Overall Architecture

GenePheno accepts two types of inputs: (1) gene sequence embeddings \(\mathbf{e}_i\) extracted by the GENERator encoder; and (2) fine-grained GO functional embeddings \(\mathbf{h}_i\) generated by the GoBERT encoder. The two modalities are fused via a cross-attention mechanism, followed by attentive pooling and concatenation to produce multimodal embeddings \(\mathbf{x}_i\), which are then passed through an MLP and a GO bottleneck layer for multi-label phenotype prediction.

The overall training objective is: $\(\mathcal{L} = \mathcal{L}_{\text{MLC}}(\hat{\mathbf{y}}, \mathbf{y}) + \lambda_1 \mathcal{L}_{\text{ex}}(\hat{\mathbf{y}}, \mathbf{y}, \mathcal{E}) + \lambda_2 \mathcal{L}_{\text{MLC}}^{\text{GO}}(\hat{\mathbf{g}}, \mathbf{g})\)$

Key Designs

1. Contrastive Multi-Label Learning Objective:

   • Function: Replaces the standard BCE loss; models multi-label prediction using a contrastive loss derived from InfoNCE.
   • Mechanism: Logits corresponding to positive and negative labels are treated as positives and negatives in contrastive learning. The positive term \(\mathcal{L}_{\text{NCE}}^+ = \sum_{i \in \Omega_+} \log(e^{s_i/\tau} + \sum_{j \in \Omega_-} e^{-s_j/\tau}) - s_i/\tau\) pulls positive label clusters closer while pushing away negative labels; the negative term \(\mathcal{L}_{\text{NCE}}^-\) handles negative labels symmetrically.
   • Design Motivation: BCE loss assumes label independence and introduces high-order exponential terms that exacerbate class imbalance. The contrastive loss implicitly models label dependencies (via global pooling of positive and negative logits) and degenerates to the ZLPR loss at \(\tau=1\), inheriting its robustness to class imbalance.
   • Distinction from Prior Work: This represents a principled extension of unsupervised contrastive learning (InfoNCE) to the supervised multi-label setting, rather than an empirical design choice.

2. Phenotype Exclusivity Regularization:

   • Function: Automatically identifies mutually exclusive phenotype pairs in the phenotype ontology using an LLM, and applies a regularization term to prevent the model from simultaneously predicting exclusive phenotypes.
   • Mechanism: For each mutually exclusive phenotype pair \((i,j) \in \mathcal{E}\), a Softplus regularization is applied: \(\mathcal{L}_{\text{ex}} = \frac{1}{N}\sum_{n=1}^N \sum_{(i,j)\in\mathcal{E}} \log(1 + e^{s_i(\mathbf{x}_n) + s_j(\mathbf{x}_n)})\)
   • Theoretical Guarantee (Proposition 1): At first-order stationary points, at least one logit in each exclusive pair satisfies \(\leq 0\), ensuring that two mutually exclusive phenotypes are not predicted simultaneously.
   • Theoretical Guarantee (Theorem 1): The exclusivity regularization simultaneously serves as implicit norm regularization, providing a generalization bound no worse than that of the unregularized loss; furthermore, the probability of conflicting predictions decays at a rate no slower than the generalization error.
   • Design Motivation: Mutual exclusions in phenotype ontologies (e.g., hypotonia vs. hypertonia in HPO) encode important biological priors that standard contrastive losses cannot capture.

3. Gene Function Bottleneck Layer:

   • Function: Places a bottleneck at the penultimate layer of the network, whose \(n\) designated nodes serve as predictions for coarse-grained GO functions.
   • Mechanism: Fine-grained GO terms (nodes at depth \(d > 2\)) are used as input and fused with sequence embeddings via cross-attention; coarse-grained GO terms (nodes at depth \(d = 2\)) serve as supervision targets for the bottleneck layer via an additional contrastive loss \(\mathcal{L}_{\text{MLC}}^{\text{GO}}\).
   • Design Motivation: Dual role — (a) during training, biological functional information guides end-to-end learning; (b) during inference, the connection weights \(w_{ij}\) of the bottleneck layer quantify associations between GO functions and phenotypes, providing interpretable mechanistic insights.
   • Distinction from Prior Work: Methods such as DeepPheno treat GO functions as input features, whereas GenePheno uses them as intermediate supervision signals, modeling the information pathway from sequence to function to phenotype.

4. Dual-Granularity GO Information Integration:

   • Function: Uses fine-grained GO embeddings at the input layer and coarse-grained GO labels at the bottleneck layer.
   • Mechanism: Fine-grained GO terms with experimentally supported annotations are retrieved from UniEntrezDB; embeddings are generated by GoBERT and fused with sequence representations via cross-attention. Coarse-grained GO terms are represented as binary vectors propagated along "is_a" and "part_of" relations in the GO DAG.
   • Design Motivation: Fine-grained functional information provides rich semantic content to assist feature extraction, while coarse-grained functions represent general biological mechanisms and are well-suited as interpretable intermediate concepts.

Loss & Training

The total loss \(\mathcal{L} = \mathcal{L}_{\text{MLC}} + \lambda_1 \mathcal{L}_{\text{ex}} + \lambda_2 \mathcal{L}_{\text{MLC}}^{\text{GO}}\) comprises three terms corresponding to phenotype prediction, exclusivity constraint, and functional bottleneck supervision, respectively. The temperature parameter \(\tau\) controls the sharpness of the contrastive loss.

Key Experimental Results

Main Results

Evaluation is conducted on four datasets (MPO, HPO, GWAS, CAFA2 wPPI), stratified by phenotype frequency (11–30, 31–100, 101–300, ≥301).

Dataset	Metric	GenePheno	Prev. SOTA	Gain
MPO	\(F_{\max}\) (All)	31.14	29.66 (InterLabelGO)	+1.48
MPO	AUC (All)	67.86	59.89 (DeepPheno)	+7.97
HPO	\(F_{\max}\) (All)	43.17	40.45 (InterLabelGO)	+2.72
HPO	AUC (All)	71.44	58.17 (DeepPheno)	+13.27
GWAS	\(F_{\max}\) (All)	40.54	37.83 (InterLabelGO)	+2.71
GWAS	AUC (All)	56.34	53.53 (kmer2Vec)	+2.81
CAFA2 wPPI	\(F_{\max}\) (All)	37.94	37.55 (DeepPheno)	+0.39
CAFA2 wPPI	AUC (All)	62.53	58.11 (GraphPheno)	+4.42

Ablation Study

Configuration	HPO \(F_{\max}\)	HPO AUC	MPO \(F_{\max}\)	MPO AUC
Full model	43.17	71.44	31.14	67.86
w/o GO input	39.96	56.60	28.21	54.19
w/o sequence input	40.99	60.23	29.13	64.72
w/o contrastive loss	41.83	65.75	29.99	64.64
w/o exclusivity loss	43.07	65.22	29.92	62.92
w/o bottleneck loss	42.94	68.17	30.13	64.67

Key Findings

• Removing the contrastive loss causes the largest performance drop, validating the importance of modeling inter-phenotype correlations.
• Removing GO input has the greatest impact on AUC (HPO: from 71.44 to 56.60), indicating that functional information is a core feature.
• Using only GO (without sequence) achieves an AUC of 64.72 on MPO but only \(F_{\max}\) of 29.13, demonstrating the complementarity of sequence and GO information.
• Exclusivity loss and bottleneck loss each contribute consistent but moderate gains.
• GenePheno demonstrates significant advantages on low-frequency phenotypes (11–30 occurrences), suggesting that the contrastive loss effectively mitigates class imbalance.
• In case studies, bottleneck weights reveal biologically plausible function–phenotype associations (e.g., isomerase activity with energy metabolism, postsynaptic membrane with muscle weakness).

Highlights & Insights

• GenePheno is the first end-to-end framework for directly predicting large-scale phenotype abnormalities from gene sequences, bridging the modality gap between sequence and phenotype.
• The derivation from InfoNCE to a multi-label contrastive loss is principled and well-motivated, rather than being a purely empirical design.
• The exclusivity regularization is backed by rigorous theoretical analysis (stationary point analysis, generalization bound, and conflict probability bound), rather than serving as a mere heuristic trick.
• The use of LLMs to automatically mine mutually exclusive phenotype pairs from ontologies is methodologically novel and extensible to other ontologies.
• The GO bottleneck layer simultaneously serves performance and interpretability, representing a key design highlight.

Limitations & Future Work

• Gains on smaller datasets such as GWAS are relatively modest (\(F_{\max}\) gain of only +2.71).
• The CAFA2 dataset requires PPI information to ensure fair comparison, limiting the scope of comparison with methods that do not rely on PPI.
• The quality of mined exclusive pairs depends on LLM judgment, which may introduce noise.
• Interpretability at inference time relies primarily on bottleneck weight heatmaps, which still require domain expert validation.
• The gene sequence encoder (GENERator) is a pretrained model, and end-to-end fine-tuning incurs substantial computational overhead.
• The current framework only considers single-gene knockouts; combinatorial effects of multi-gene knockouts are not addressed.

Related Work & Insights

• The paradigm of deriving multi-label loss functions from InfoNCE can be directly generalized to other multi-label classification tasks with label correlations.
• The methodology of exclusivity regularization combined with LLM-mined constraints is applicable to any classification problem with ontological structure (e.g., disease classification, species classification).
• The idea of using a bottleneck layer to provide interpretable intermediate concepts draws inspiration from Concept Bottleneck Models (CBM), with a natural biological correspondence (GO functions) in this setting.
• Curated-data-dependent methods such as DeepPheno and SSLPheno serve as comparative baselines; GenePheno's direct sequence input approach offers a fundamental scalability advantage.

Rating

• Novelty: ⭐⭐⭐⭐⭐
• Experimental Thoroughness: ⭐⭐⭐⭐⭐
• Writing Quality: ⭐⭐⭐⭐⭐
• Value: ⭐⭐⭐⭐

Related Papers

• [ACL 2026] Preference Heads in Large Language Models: A Mechanistic Framework for Interpretable Personalization
• [ICLR 2026] I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
• [ICLR 2026] When Thinking Backfires: Mechanistic Insights into Reasoning-Induced Misalignment
• [ACL 2026] IDEA: An Interpretable and Editable Decision-Making Framework for LLMs via Verbal-to-Numeric Calibration
• [ICML 2026] Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions