Skip to content

EquaCode: A Multi-Strategy Jailbreak Approach for Large Language Models via Equation Solving and Code Completion

Conference: AAAI 2026 arXiv: 2512.23173
Code: GitHub
Area: Code Intelligence Keywords: Jailbreak Attack, LLM Safety, Equation Solving, Code Completion, Cross-Domain Attack

TL;DR

This paper proposes EquaCode, a multi-strategy jailbreak method that decomposes malicious queries into a cross-domain combination of equation solving (\(B+C+x=A\)) and code completion (completing the solve() method of a Solver class), achieving an average attack success rate of 92.78% on the GPT series and approaching 100% on the latest models (Gemini/DeepSeek/Grok).

Background & Motivation

State of the Field

Background: LLM safety alignment primarily defends against natural language inputs, leaving significant vulnerabilities in non-natural-language domains (mathematics, code).

Limitations of Prior Work: (a) White-box attacks require model gradients and are impractical; (b) Black-box attacks (GCG, AutoDAN) require multiple iterations; (c) Single-domain attacks (equation-only or code-only) achieve limited success rates.

Key Challenge: The very capabilities of LLMs in mathematical reasoning and code completion become weaknesses for bypassing safety alignment—safety training is concentrated on natural language.

Goal: To expose cross-domain security vulnerabilities in LLMs and achieve high-success-rate black-box jailbreaks in a single query.

Key Insight: Exploiting the synergy between two non-natural-language domains—equations and code—where equations structure the malicious query as a variable-solving problem and code wraps the equation as a functional completion task.

Core Idea: The cross-domain combination of the equation domain and the code domain produces a synergistic amplification effect that breaks through the protections of natural language safety alignment.

Method

Overall Architecture

Input: malicious query. Output: harmful response generated by the LLM. Pipeline: (1) the Equation Module decomposes the query into \(B\) (subject) \(+ C\) (tool) \(+ x\) (execution steps) \(= A\) (objective); (2) the Code Module wraps the equation inside a Solver class and requests completion of the solve() function.

Key Designs

  1. Equation Module:

    • Function: Structures the malicious query as an equation-solving problem.
    • Mechanism: Decomposes the query into \(B + C + x = A\), where \(B\) is the subject, \(C\) is the tool, and \(x\) is the execution steps to be solved by the LLM.
    • Design Motivation: The LLM's mathematical reasoning mode bypasses natural language safety checks, and the equation form conceals malicious intent.

  2. Code Module:

    • Function: Wraps the equation in code to trigger the code completion mode.
    • Mechanism: Encapsulates equation components in a Solver class with an empty steps list and an empty solve() method, prompting the LLM to complete it.
    • Design Motivation: Code completion mode shifts the LLM's focus toward syntactic correctness rather than content safety.

  3. Cross-Domain Synergistic Amplification:

    • Function: The combined effect of equation + code exceeds the sum of their individual effects.
    • Mechanism: Equations provide structured decomposition while code provides functional wrapping; together they make safety mechanisms harder to trigger.
    • Design Motivation: Single-domain attacks are more easily detected; cross-domain combination creates stronger obfuscation.

Loss & Training

No training required—purely black-box attack with a single query (no iterative refinement). A total of 520 queries from the AdvBench dataset are used.

Key Experimental Results

Main Results (AdvBench Attack Success Rate)

Model Attack Success Rate (%)
GPT-4 91.92
GPT-4-Turbo 98.46
GPT-3.5-Turbo 97.12
GPT-4o 87.12
GPT Series Average 92.78
Gemini-1.5 100.00
DeepSeek-R1 100.00
DeepSeek-v3 100.00
Grok-3 95.96
Latest Models Average 98.99
Llama-3.1-405B 17.88

Ablation Study

Configuration Performance Note
EquaCode (Full) Optimal Equation + Code synergy
Equation Only Significantly below full Lacks code wrapping
Code Only Significantly below full Lacks structured decomposition
Synergistic Effect > Sum of individual Confirms cross-domain amplification

Key Findings

  • Severe cross-domain security vulnerabilities: The latest models (Gemini-1.5/DeepSeek-R1/v3) are compromised at nearly 100%.
  • High single-query success rate: No iteration required; 520 queries suffice for systematic evaluation.
  • Anomalous performance of Llama: Only 17.88% success rate, possibly due to its weaker comprehension of equation formats.
  • The synergistic effect of equation + code exceeds the sum of individual effects, confirming the efficacy of cross-domain combination.

Highlights & Insights

  • Exposes security blind spots in non-natural-language domains: Reveals a structural weakness in current LLM safety alignment. Safety training is overly focused on natural language, leaving mathematical and code domains insufficiently protected.
  • Efficiency of single-query black-box attacks: No iterative optimization or model gradients are required, making the practical threat more significant.
  • Quantitative demonstration of synergistic effects: Ablation experiments clearly show that cross-domain combination outperforms single-domain attacks.

Limitations & Future Work

  • No defense mechanism is proposed: The paper only exposes vulnerabilities without discussing remediation.
  • The anomalously low success rate on Llama is not analyzed in depth.
  • The AdvBench dataset is relatively small (520 samples) and may not represent all types of malicious queries.
  • Future work should investigate how to incorporate equation and code domains into safety alignment training.
  • vs. GCG/AutoDAN: These methods require multiple iterations, whereas EquaCode succeeds in a single query with higher success rates.
  • vs. Natural Language Jailbreaks (e.g., DAN): Natural language jailbreaks rely on role-playing, while EquaCode leverages domain shifting.
  • Insight: LLM safety alignment must cover all functional domains, not just natural language.

Rating

  • Novelty: ⭐⭐⭐⭐ The cross-domain jailbreak approach is novel; exploiting mathematical equation encoding to bypass safety alignment is unexpected.
  • Experimental Thoroughness: ⭐⭐⭐⭐ Multi-model, multi-ablation evaluation, with effectiveness validated on closed-source models.
  • Writing Quality: ⭐⭐⭐⭐ Method description is clear and the attack pipeline is reproducible.
  • Value: ⭐⭐⭐⭐⭐ Reveals an important security vulnerability—LLMs' mathematical processing capabilities can be exploited to circumvent text-level safety checks—with direct implications for security research.

Additional Notes

  • Equation-based encoding is a novel semantic-preserving jailbreak strategy that is difficult to detect with existing defenses (e.g., perplexity filtering), necessitating a multimodal security auditing framework.