---

title: >- [Paper Note] Instant Expressive Gaussian Head Avatars at Over 100 FPS description: >- [ECCV2026][3D Vision][3D Gaussian Splatting] An instant feedforward Gaussian head avatar animation framework using decoupled motion basis vectors and high-dimensional feature-space deformation, running at over 100 FPS with fine-grained expressions and strict 3D consistency. tags: - ECCV2026 - 3D Vision - 3D Gaussian Splatting - Head Avatar - Real-Time Portrait Animation date: 2026-09-19 content_hash: 5df2160bda5a3fcd

Instant Expressive Gaussian Head Avatars at Over 100 FPS

Conference: ECCV 2026
Paper: CVF Open Access
Area: 3D Vision
Keywords: 3D Gaussian Splatting, Head Avatar, Feedforward Encoder, Feature-Space Deformation, Real-Time Portrait Animation

TL;DR

This paper tackles the portrait animation trilemma among inference speed, 3D multi-view consistency, and facial expressiveness by introducing a feedforward Gaussian framework that deforms primitives in a high-dimensional feature space using decoupled motion basis vectors, achieving detailed expression transfer and strict 3D consistency at 107.31 FPS.

Background & Motivation

Synthesizing a controllable 4D digital twin from a single portrait image that supports both free-viewpoint navigation and dynamic facial expression reenactment is a cornerstone for applications across VR/AR, telepresence, and agentic AI. Recently, 2D facial animation models driven by video diffusion backbones have set new standards for photorealistic expressiveness, rendering subtle facial dynamics like nasal wrinkles and lip nuances faithfully. However, these 2D diffusion pipelines require iterative multi-step denoising that is computationally prohibitive for real-time interactive systems, and their lack of an underlying 3D geometry inevitably induces multi-view inconsistencies, geometric distortion, and temporal flickering under large head pose rotations.

To achieve view consistency, feedforward 3D portrait generation has explored neural radiance fields (NeRFs) and 3D Gaussian splatting (3DGS). Earlier 3D approaches heavily depend on 3D morphable models (3DMM or FLAME) as parametric priors, which inherently restrict expressiveness to a predefined linear subspace and struggle with asymmetric micro-expressions or personalized wrinkles. More recent data-driven 3D implicit works attempt to overcome this by predicting global residual triplanes or feature maps; however, they rely on computationally intensive dense attention mechanisms to repeatedly fuse 3D structure and animation cues across the entire volume at every animation step, bottlenecking inference speed to a few frames per second. As a result, existing portrait animation methods remain constrained by a fundamental trilemma: sacrificing either speed, 3D multi-view consistency, or expressive motion fidelity.

This paper breaks through this trade-off with a key insight: rather than iteratively recalculating dense global triplane features via heavy cross-attentions, static 3D appearance and dynamic motion can be completely decoupled onto discrete 3D Gaussian primitives, allowing each primitive to deform independently in a high-dimensional feature space guided by personalized motion bases. Core idea: model facial dynamics as local residual updates on decoupled motion basis vectors within a high-dimensional feature space, trained via frontalized diffusion priors and multi-view distillation, delivering high-fidelity 3D-consistent portrait animation at 107.31 FPS.

Method

Overall Architecture

The pipeline consists of a single-view 3D Gaussian reconstruction stage and an instant feature-space animation stage. During reconstruction, a feedforward lifting encoder maps an input source portrait \(I_s\) into a 96-channel neural triplane. Using ray shooting and opacity-guided importance sampling, approximately 200K 3D Gaussian primitives are sampled. For each primitive, the first 48 channels provide static geometry and appearance features \(f_i\), while the remaining 48 channels encode an individual motion basis vector \(m_i\). During animation, a frozen motion encoder extracts compact 1D motion coefficients from the source and driving images, which condition a lightweight motion decoder via AdaLN to predict a residual feature \(\delta f_i\) for each Gaussian independently. The perturbed feature is decoded into 3D Gaussian attributes by an MLP, and the driven avatar is rendered from any target viewpoint via differentiable Gaussian splatting rasterization.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Single source portrait $I_s$"] --> B["Triplane lifting & motion basis decoupling<br/>Encode 96-channel triplanes & sample static features $f_i$ and motion bases $m_i$"]
    C["Driving image $I_d$ & source image $I_s$"] --> D["Compact motion coefficient extraction<br/>Frozen motion encoder $M$ outputs 1D coefficients $M(I_s), M(I_d)$"]
    B --> E["High-dimensional feature-space deformation<br/>Modulate motion basis $m_i$ via AdaLN & predict residual $\delta f_i$"]
    D --> E
    E --> F["Gaussian attribute non-linear decoding & rasterization<br/>Decode Gaussian position/scale/rotation/opacity/color & splat to $I'_d$"]
    F --> G["Driven novel-view output (107.31 FPS)"]

Key Designs

1. Triplane-based Gaussian sampling with decoupled motion basis vectors: eliminating repetitive 3D structure lifting Existing implicit 3D animation frameworks (e.g., Portrait4D or VOODOO-XP) update or recompute global 3D feature grids across the full face volume for every driving frame, causing severe computational overhead. In this work, the authors adapt the LP3D lifting backbone to map the source image into 96-channel neural triplanes \(\{T_{xy}, T_{yz}, T_{zx}\}\). By shooting rays from a fixed canonical frontal camera and performing two-pass importance sampling, approximately 200K spatial sample points \(x_i\) are instantiated. Querying the triplane yields a 48-dimensional static appearance feature \(f_i\) from the first 48 channels and a 48-dimensional motion basis vector \(m_i\) from the remaining 48 channels. Analogous to localized blendshape bases or localized facial muscles, \(m_i\) encapsulates subject-specific dynamic responsiveness (e.g., wrinkle formation tendency) without altering the underlying spatial structure. Crucially, \(m_i\) remains constant throughout an animation sequence, meaning the expensive 2D-to-3D lifting step is executed only once per identity (taking merely 20 ms).

2. High-dimensional feature-space local deformation: overcoming expressiveness limits of low-dimensional physical warping Conventional dynamic 3D Gaussian methods typically apply deformation directly in low-dimensional Euclidean space by predicting offsets for 3D coordinates \(\Delta \mu\), rotation quaternions \(\Delta q\), or scales \(\Delta s\). However, such physical-space deformation struggles to capture non-linear expression details, subtle skin folds, or shading variations caused by muscle contractions. Instead, this paper performs deformation in the high-dimensional feature space. A pretrained, frozen motion encoder \(M\) extracts 1D motion vectors \(M(I_s), M(I_d) \in \mathbb{R}^{512}\) from the source and driving images. Their concatenation conditions a single-layer Adaptive Layer Normalization (AdaLN) block to modulate each primitive's motion basis vector \(m_i\). A lightweight MLP motion decoder \(\Psi\) with one hidden layer (96 units) then predicts the residual feature vector \(\delta f_i\):

\[\Phi(f_i + \delta f_i(I_s \to I_d)) = \{\mu_i, s_i, q_i, o_i, c_i\}\]

The updated feature vector is mapped to Gaussian attributes—position \(\mu_i\), scale \(s_i\), rotation quaternion \(q_i\), opacity \(o_i\), and color \(c_i\)—using a compact attribute decoder \(\Phi\) (a single-layer MLP with softplus activations). Because every Gaussian primitive is updated independently via local modulation without dense global attention layers across the volume, the computational cost per frame is drastically reduced, while the high-dimensional latent manifold easily models non-linear surface dynamics and fine wrinkles.

3. Frontalized diffusion distillation & multi-view geometric supervision: combining generative expressiveness with 3D consistency To train the model on diverse, extreme facial expressions without relying on scarce multi-view 4D video scans, the authors leverage X-NeMo, a 2D diffusion facial animation model, to synthesize over 60,000 identities from FFHQ with 8 extreme expressions each. However, naively supervising 3D models with 2D diffusion outputs leads to severe multi-view hallucinations and identity drift under varying camera poses. The method decouples geometry from expression: portraits are first strictly frontalized using LP3D, and X-NeMo is applied solely to frontal viewpoints to prevent diffusion-induced shape distortions. During training, a frozen multi-view estimator generates novel viewpoints dynamically from the frontalized outputs, providing strict geometric supervision and ensuring the synthesized 3D avatar remains completely free of 2D generative hallucinations.

Loss & Training

The framework is optimized end-to-end via a self-reenactment objective combining pixel-level and perceptual supervision:

\[\mathcal{L} = \lambda_{\text{L1}}\mathcal{L}_{\text{L1}}(I'_d, I_d) + \lambda_{\text{LPIPS}}\mathcal{L}_{\text{LPIPS}}(I'_d, I_d) + \lambda_{\text{ID}}\mathcal{L}_{\text{ID}}(I'_d, I_d) + \lambda_{\text{Detail}}\mathcal{L}_{\text{Detail}}(I'_d, I_d) + \lambda_{\text{Norm}}\mathcal{L}_{\text{Norm}} + \lambda_{\text{adv}}\mathcal{L}_{\text{adv}}\]

Hyperparameters are configured as \(\lambda_{\text{L1}} = 1.0\), perceptual loss \(\lambda_{\text{LPIPS}} = 1.0\), identity preservation loss \(\lambda_{\text{ID}} = 0.1\), and local detail loss \(\lambda_{\text{Detail}} = 0.1\) (applying targeted L1 penalties to the eye and mouth regions). The term \(\lambda_{\text{Norm}} = 0.001\) penalizes the average L2 norm of predicted residuals \(\delta f_i\) to enforce motion sparsity, and \(\lambda_{\text{adv}} = 0.025\) introduces an adversarial discriminator conditioned on motion coefficients. Networks are trained with the Adam optimizer at a learning rate of \(5\times 10^{-5}\) and batch size 32, progressively scaling the rendering resolution from 64 to 512.

Key Experimental Results

Main Results

Quantitative evaluation on the VOODOO-XP test set for cross-identity reenactment (Cross-reenactment, corresponding to Table 2 in the paper) across 2D diffusion/GAN baselines and 3D methods (MEt3R measures 3D dense inconsistency, EMO measures emotion similarity, AED measures average FLAME expression distance, APD measures pose error, and FPS is benchmarked on an NVIDIA 6000 Ada GPU):

Category	Method	MEt3R ↓	ID ↑	EMO ↑	AED ↓	APD ↓	FPS ↑
2D	LivePortrait	0.033	0.74	0.716	0.810	0.026	78.12
2D	HYPortrait	0.032	0.74	0.752	0.900	0.077	0.01
2D	X-NeMo	0.035	0.72	0.760	0.805	0.032	0.03
3D	GAGAvatar	0.034	0.77	0.654	0.888	0.025	0.41
3D	InvertAvatar	0.028	0.80	0.565	0.891	0.049	0.07
3D	VOODOO-XP	0.032	0.77	0.699	0.903	0.028	5.45
3D	Portrait4D-v2	0.035	0.79	0.589	0.886	0.029	14.50
3D (Ours)	Ours	0.028	0.76	0.785	0.731	0.028	107.31

Note: In Table 2, the proposed method achieves the lowest expression error AED (0.731) and the highest emotion score EMO (0.785), outperforming both 3D baselines and 2D diffusion models while operating at 107.31 FPS.

Ablation Study

Ablation experiments on self-reenactment on the VOODOO-XP test set at \(128 \times 128\) resolution without adversarial loss (Table 3 in the paper):

Configuration	MEt3R ↓	PSNR ↑	SSIM ↑	LPIPS ↓	AED ↓	IoU ↑	Memory ↓	FPS ↑
Ours Full Model	0.073	23.18	0.785	0.104	0.507	0.88	0.32 GB	132.87
w/ DINO-v2 Motion Encoder	0.074	22.88	0.776	0.110	0.597	0.87	4.32 GB	22.47
w/ Real Dataset (CelebVText)	0.073	23.10	0.782	0.109	0.543	0.88	0.32 GB	132.87
w/ Spatial Deformation	0.075	23.27	0.788	0.109	0.634	0.89	0.27 GB	141.61

Key Findings

• Feature-space deformation is essential for expressive micro-dynamics: Directly predicting 3D spatial coordinate offsets (Spatial Deformation) slightly speeds up inference to 141.61 FPS by bypassing the non-linear decoder, but severely degrades expression accuracy, causing AED to rise from 0.507 to 0.634 and smoothing out wrinkles.
• Diffusion-synthesized extreme expressions prevent expression damping: Substituting the synthetic training set with a real video corpus (CelebVText) leads to muted reenactment results under extreme driving expressions, increasing AED to 0.543 due to the scarcity of high-intensity expression data in wild videos.
• High throughput and minimal memory footprint: Decoupling the per-frame motion prediction to a single lightweight MLP operating on fixed Gaussian primitives requires only 0.32–0.4 GB of GPU memory and reaches 107.31 FPS end-to-end, providing a 3 to 4 orders of magnitude speedup over diffusion-based portrait models (0.01–0.03 FPS).

Highlights & Insights

• Decoupled personalized motion basis vectors: Splitting triplane features into static geometry channels and static motion basis vectors provides each Gaussian primitive with individual deformation responsiveness, mimicking blendshapes while eliminating expensive global attention queries.
• Superiority of high-dimensional manifold deformation: Modeling dynamic facial motions within high-dimensional Gaussian feature space provides a richer, smoother manifold that naturally captures complex micro-expressions, skin folding, and shading changes.
• Frontalized diffusion distillation strategy: Constraining 2D diffusion generation to canonical frontal poses while enforcing multi-view consistency via pre-trained 3D lifting networks establishes an effective pipeline for distilling expressive 2D motion priors into 3D avatars without generative hallucination.

Limitations & Future Work

• Thin or reflective accessories: Fine structures such as eyeglasses with thin frames and complex specular reflection/refraction remain challenging to reconstruct accurately from single-view inputs.
• Lack of explicit illumination decoupling: The Gaussian primitives currently encode entangled appearance and lighting, which can cause subtle baked-in shading artifacts under drastic lighting alterations.
• Multi-modal driving signals: The current pipeline is validated on video/image-driven expressions; mapping audio waveforms or textual instructions into 1D motion coefficients represents a promising extension for full conversational agents.

Related Work & Insights

• vs LivePortrait / X-NeMo (2D portrait animation): 2D generative approaches suffer from high latency (X-NeMo at 0.03 FPS) or 3D multi-view inconsistency; this method matches or exceeds their expression expressiveness while ensuring strict multi-view geometry (MEt3R 0.028 vs X-NeMo's 0.035) at over 100 FPS.
• vs Portrait4D-v2 / VOODOO-XP (3D triplane deformation): Prior data-driven 3D methods rely on global attention across triplane grids, limiting frame rates to 5.45–14.50 FPS; this method's local primitive-level deformation delivers a 7–20× speedup.
• vs GAGAvatar / InvertAvatar (3DMM-based animation): Parametric linear blendshape priors restrict expression fidelity and fail on asymmetric or extreme wrinkles; this method's non-parametric learned representation achieves superior expression accuracy (AED 0.731 vs 0.888–0.891).

Rating

• Novelty: ⭐⭐⭐⭐⭐ (Elegant combination of decoupled motion bases, high-dimensional feature deformation, and frontalized diffusion distillation)
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Exhaustive comparisons across 2D/3D baselines, extensive ablations on motion encoders, datasets, and deformation spaces)
• Writing Quality: ⭐⭐⭐⭐⭐ (Well-structured paper with compelling motivation and insightful analysis of the portrait animation trilemma)
• Value: ⭐⭐⭐⭐⭐ (Real-time 107+ FPS performance with high expressivity and 3D consistency provides strong practical utility for digital avatars and telepresence)

Related Papers

• [CVPR 2025] 3D Gaussian Head Avatars with Expressive Dynamic Appearances by Compact Tensorial Representations
• [CVPR 2026] PhysHead: Simulation-Ready Gaussian Head Avatars
• [CVPR 2026] FastGS: Training 3D Gaussian Splatting in 100 Seconds
• [ICLR 2026] FastGHA: Generalized Few-Shot 3D Gaussian Head Avatars with Real-Time Animation
• [CVPR 2026] Multi-view Consistent 3D Gaussian Head Avatars 'without' Multi-view Generation