FPicker: Topology-Guided Evolution for Filament Tracing in Low-SNR Microscopy

Conference: ECCV 2026
Paper: ECCV 2026
Code: https://github.com/tomzhaosky/FPicker
Area: Computational Biology / Microscopic Image Analysis
Keywords: Cryo-EM, Filament Tracing, Open-Curve Evolution, Low-SNR Perception, AI for Science

TL;DR

Tsinghua University researchers present FPicker, which constructs a coarse topological skeletal prior via a box-free center-endpoint representation and iteratively deforms it through an open-boundary graph convolutional snake module with bridge curriculum training, eliminating ghost center drift and tip shrinking in low-SNR Cryo-EM filament tracing (SNR = -20 dB).

Background & Motivation

Elucidating the atomic structure of filamentous proteins, such as amyloid fibrils involved in neurodegenerative diseases, is fundamental to structural biology and pharmaceutical discovery. Cryo-Electron Microscopy (Cryo-EM) enables near-atomic observation, yet the foundational filament picking step—detecting and tracing continuous curvilinear protein backbones from noisy micrographs—remains a laborious bottleneck. Unlike globular macromolecules, protein filaments appear as continuous flexible curves that frequently overlap, run parallel, or intersect in two-dimensional projections. Furthermore, embedded in vitreous ice, micrographs exhibit severe low Signal-to-Noise Ratios (\(\text{SNR} = \sigma_s^2 / \sigma_n^2 < 0.1\), or -10 dB to -20 dB), where structural signals are almost completely masked by shot and Poisson noise.

Standard visual paradigms suffer from severe geometric incompatibilities under this extreme noise regime. Box-based detectors (e.g., crYOLO, YOLOv8) attempt to track filaments by linking bounding boxes, but encapsulating a 1D thin curve inside a 2D rigid box inevitably encloses mostly background noise; this causes geometric centers to drift into signal-free regions—termed ghost center degeneracy—and fractures trajectories. Pixel-wise segmentation models (e.g., Topaz, SAM 2) merge distinct instances at 2D crossover junctions because intersecting filaments share spatial projection pixels, while non-differentiable skeletonization cannot leverage global topological continuity to heal noise-induced gaps. Meanwhile, sequential trackers derail via Markovian error accumulation, and traditional active contour models (e.g., Deep Snake, CurveGCN) not only inherit box-induced ghost centers during initialization but also impose periodic boundary conditions that apply artificial tensile forces pulling endpoints together, causing severe shrinking degeneracy.

To reconcile these bottlenecks, the authors re-examine active contour modeling by abandoning cyclic constraints. Core idea: FPicker unifies filament tracing into a two-stage coarse-to-fine framework that predicts a box-free linear skeletal prior via a topological center-endpoint representation, and refines it via open-boundary graph convolution (OBC) with dual-stream deformation and bridge curriculum learning, resolving both ghost center collapse and tip shrinking.

Method

Overall Architecture

FPicker maps a raw Cryo-EM micrograph \(I\) to a set of precise, open skeletal curves \(\mathcal{F}\) through an end-to-end differentiable coarse-to-fine pipeline. A shared encoder-decoder backbone (e.g., DLA-34 or ResNet-50) first processes the input into a high-resolution semantic feature map \(F \in \mathbb{R}^{H/s \times W/s \times C}\) with stride \(s\). Stage I generates rigid linear skeletal priors by predicting topological centroid heatmaps alongside center-to-endpoint displacement vectors. Stage II constructs an open graph by uniformly sampling vertices along this prior, bilinearly interpolating features from \(F\), and iteratively applying open-boundary graph convolutions to snap the vertices onto true protein density ridges.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Raw Cryo-EM Micrograph<br/>(Low-SNR Input Micrograph I)"] --> B["Shared Feature Extractor<br/>(High-Res Feature Map F)"]
    B --> C["Stage I: Proposal Generation<br/>(Centroid Heatmap + Endpoint Displacements)"]
    B -.-> D["Auxiliary Geometric Fields<br/>(Distance Field D + Angle Field A Supervision)"]
    C --> E["Linear Skeletal Prior Construction<br/>(Arc-Length Median & Uniform Resampling)"]
    E --> F["Bridge Curriculum Strategy<br/>(Annealing Ground-Truth to Predicted Prior)"]
    B --> G["Stage II: Open-Curve Evolution<br/>(Vertex Bilinear Feature Sampling)"]
    F --> G
    G --> H["Dual-Stream Deformation Head<br/>(Open-Boundary GCN + Global Max-Pooling Anchor)"]
    H -->|Iterative T-step Refinement| I["Final Filament Trajectories<br/>(Continuous Open-Curve Skeletons)"]

Key Designs

1. Topology-Aware Proposal Generation: Eliminating Ghost Center Degeneracy

To circumvent the fundamental geometric mismatch of placing rigid 2D bounding boxes around thin 1D flexible filaments, FPicker eliminates bounding box regression entirely. The framework redefines the detection target as a topological centroid \(c_i = G_i(L_i / 2)\), strictly constrained to the arc-length median of the filament trajectory \(G_i(\ell)\). This anchors instance proposals onto discriminative protein density rather than background noise. Peak extraction on the predicted heatmap via a \(3 \times 3\) max-pooling operation provides stable non-maximum suppression (NMS) in continuous feature space. Simultaneously, dense displacement fields \(\{\hat{d}_{\text{start}}, \hat{d}_{\text{end}}\}\) are regressed from the center. Interpolating \(N\) equidistant vertices between \(c_i + \hat{d}_{\text{start}}\) and \(c_i + \hat{d}_{\text{end}}\) constructs a linear topological prior \(\hat{S}_{\text{init}}\) that establishes a rotation-aware capture range, ensuring even highly curved filaments reside within the subsequent deformation receptive field.

2. Open-Boundary Graph Convolution: Eliminating Shrinking Degeneracy

Standard deep active contour models employ circular convolutions that enforce periodic boundaries by connecting vertex \(v_N\) back to \(v_1\). For open biological filaments, this cyclic constraint generates artificial tensile forces that pull the tips toward one another, causing severe tangent distortion and tip retraction. FPicker formulates Open-Boundary Graph Convolution (OBC) by applying replicate padding at graph boundaries. For a graph signal \(f \in \mathbb{R}^N\), vertex feature aggregation is strictly restricted to geodesic neighbors:

\[\mathbf{f}'_i = \sum_{j=-d}^{d} \mathbf{f}_{\max(1, \min(N, i+j))}\]

This boundary truncation completely cuts off gradient flow between logical extremities, zeroing tip tension and ensuring full-length reconstruction without shrinking.

3. Dual-Stream Structural Evolution and Auxiliary Fields: Anchoring Under Noise

Under extreme low SNR, local vertex features are easily corrupted by random noise, inducing drift. FPicker introduces a dual-stream deformation architecture: the local stream aggregates neighbor affinities through stacked OBC layers (\(\mathbf{h}_i^{\text{local}}\)), while the global stream applies global max-pooling across all \(N\) vertices to isolate the strongest structural signal (\(\mathbf{h}^{\text{global}} = \max_{j=1}^N \mathbf{f}_j^{(t)}\)). This data-dependent global descriptor acts as a semantic anchor, hallucinating trajectories for noise-submerged vertices via:

\[\Delta \hat{\mathbf{v}}_i^{(t)} = \mathbf{W}_{\text{offset}} \begin{bmatrix} \mathbf{h}_i^{\text{local}} \\ \mathbf{h}^{\text{global}} \end{bmatrix} + \mathbf{b}_{\text{offset}}, \quad \hat{\mathbf{v}}_i^{(t+1)} = \hat{\mathbf{v}}_i^{(t)} + \Delta \hat{\mathbf{v}}_i^{(t)}\]

To prevent the backbone from overfitting high-frequency Poisson noise, an auxiliary geometric branch regresses a Distance Field \(D\) (Euclidean proximity to the nearest filament) and a 2-channel Angle Field \(A = (\cos\theta, \sin\theta)\). These fields serve as continuous low-pass regularizers, rendering the shared representation resilient to Contrast Transfer Function (CTF) oscillations.

4. Bridge Curriculum Strategy: Decoupling Deformation Learning from Localization Variance

Joint end-to-end training under extreme noise triggers a cold-start dilemma: the evolution module requires a stable initial prior to master local deformation mechanics, but early-stage proposals are noisy and chaotic, trapping the GCN in local minima. The bridge curriculum strategy modulates the input skeleton \(\mathcal{S}_{\text{input}}\) using a transition regulator \(P_{\text{gt}}\):

\[\mathcal{S}_{\text{input}} = \beta (\mathcal{S}_{\text{gt}} + \epsilon) + (1 - \beta) \hat{\mathcal{S}}_{\text{pred}}, \quad \beta \sim \text{Bernoulli}(P_{\text{gt}})\]

During the initial 10 epochs, \(P_{\text{gt}} = 1\) injects ground-truth skeletons perturbed with Gaussian noise \(\epsilon\), enabling the GCN to master feature attraction (snapping to ridge intensity) within a controlled capture range. Over the subsequent 110 epochs, \(P_{\text{gt}}\) is linearly annealed to 0, gradually exposing the evolution module to predicted prior variance until fully autonomous inference is achieved.

Loss & Training

The network optimizes a unified multi-task objective:

\[\mathcal{L}_{\text{total}} = \lambda_{\text{hm}} \mathcal{L}_{\text{hm}} + \lambda_{\text{reg}} \mathcal{L}_{\text{reg}} + \lambda_{\text{ends}} \mathcal{L}_{\text{ends}} + \lambda_{\text{aux}} \mathcal{L}_{\text{aux}} + \lambda_{\text{evol}} \mathcal{L}_{\text{evol}}\]

• Centroid Heatmap Loss (\(\mathcal{L}_{\text{hm}}\)): A modified Focal Loss (\(\alpha=2, \beta=4\)) anchored onto topological centroids \(c_i = G_i(L_i / 2)\).
• Sparse Regression Losses (\(\mathcal{L}_{\text{reg}}\) and \(\mathcal{L}_{\text{ends}}\)): L1 losses computed strictly at ground-truth centroid positions \(c_k\) for sub-pixel center offset \(\hat{o}\) and endpoint displacement vectors \(\hat{d}\), avoiding background noise overfitting.
• Auxiliary Geometric Loss (\(\mathcal{L}_{\text{aux}}\)): Dense L1 loss over the valid fiber mask between predicted fields \((\hat{D}, \hat{A})\) and ground truth.
• Open-Curve Evolution Loss (\(\mathcal{L}_{\text{evol}} = \mathcal{L}_{\text{fit}} + \lambda_{\text{uni}} \mathcal{L}_{\text{uni}}\)): Smooth-L1 alignment loss \(\mathcal{L}_{\text{fit}}\) between evolved vertices and ground truth, combined with an edge-length variance penalty \(\mathcal{L}_{\text{uni}}\) to prevent vertex clustering and enforce equidistant vertex distributions.

Key Experimental Results

Main Results

On the physics-based Cryo-Sim benchmark (simulating electron optics and CTF parameters across High, Medium, and Extreme noise), FPicker was evaluated against box detectors, pixel-wise segmenters, and closed active contours. Performance is quantified using mean spatio-angular precision (mSAP, with angular tolerance \(\Delta\theta < 15^\circ\)), clDice, fragmentation-penalized clDice (fp-clDice), Gap Rate (\(1 - \text{mAR}\)), and penalized angle error (P-Ang).

Table 1: Main Comparison across SNR regimes on Cryo-Sim (Excerpt from Table 1 in original paper)

Paradigm	Method	Medium SNR (-13 dB) mSAP (%) ↑	Medium SNR (-13 dB) Gap (%) ↓	Extreme SNR (-20 dB) mSAP (%) ↑	Extreme SNR (-20 dB) Gap (%) ↓	Extreme SNR (-20 dB) P-Ang (°) ↓
Box/Vector	crYOLO	48.6	24.5	25.4	48.2	64.4
Box/Vector	YOLOv8	52.4	20.2	28.5	45.2	62.1
Segmentation	Topaz	48.1	24.5	24.2	48.5	65.4
Segmentation	U-Net	62.1	18.5	38.2	38.5	55.2
Segmentation	SegFormer	64.8	16.2	41.5	35.4	52.8
Segmentation	SAM 2	35.4	52.6	12.1	75.2	82.5
Active Contour	Deep Snake	68.2	12.5	45.8	28.5	45.2
Active Contour	CurveGCN	71.5	11.8	48.2	26.2	42.8
Ours	FPicker (ResNet-50)	89.0	7.1	72.1	12.2	24.5
Ours	FPicker (Swin-T)	90.5	7.3	74.8	10.5	21.8
Ours	FPicker (DLA-34)	91.1	6.3	76.5	9.8	20.5

Ablation Study

Ablation experiments on Cryo-Sim and the real-world Custom-EMPIAR benchmark progressively assess Initialization (Box vs. Center-Endpoint C-E), Topology (Closed vs. Open), Bridge Curriculum, and Auxiliary Geometric Loss.

Table 2: Component ablation study across Cryo-Sim and Custom-EMPIAR (Excerpt from Table 4 in original paper)

Model	Init	Topo	Bridge	Aux	Med SNR (-13 dB) mSAP (%) ↑	Ext SNR (-20 dB) mSAP (%) ↑	Ext SNR (-20 dB) Gap (%) ↓	EMPIAR Fine-Tuned mSAP (%) ↑	EMPIAR Fine-Tuned P-Ang (°) ↓
A (Base)	Box	Closed	-	-	55.4	2.5	92.5	24.5	75.2
B	C-E	Closed	-	-	68.5	12.5	65.4	52.5	54.8
C	C-E	Open	-	-	70.2	35.6	58.5	55.2	42.3
D	C-E	Open	-	✓	82.5	45.2	35.8	71.4	30.5
E (Full)	C-E	Open	✓	✓	91.1	76.5	9.8	82.9	13.7

Key Findings

• Catastrophic Failure of Box Priors: Under extreme noise (-20 dB), the baseline box-initialized closed snake (Model A) collapses to 2.5% mSAP with a 92.5% gap rate due to ghost center drift. Introducing the center-endpoint prior (Model B) immediately improves EMPIAR fine-tuned mSAP from 24.5% to 52.5% and drops the extreme gap rate to 65.4%.
• Eliminating Tip Tension with Open Curves: Transitioning from closed to open convolutions (Model B to C) sharply reduces P-Ang from 45.2° to 19.4° at Medium SNR and from 54.8° to 42.3° on EMPIAR, validating that breaking periodic boundary conditions resolves tip retraction.
• Bridge Curriculum Enables Extreme Noise Convergence: Comparing Model D and Full Model E, bridge curriculum annealing surges extreme SNR mSAP from 45.2% to 76.5% (+31.3% absolute gain), proving that shielding initial deformation learning from noisy proposals is critical for convergence.
• Remarkable Few-Shot Sim-to-Real Efficiency: When fine-tuning on Custom-EMPIAR, FPicker achieves an mSAP of 55.4% using only 10 annotated micrographs (nearly double its zero-shot 28.6%), and reaches 70.1% with 50 micrographs, demonstrating that the learned geometric backbone transfers efficiently to real microscope imaging.

Highlights & Insights

• Geometric Manifolds Outperform Texture Relying Models: Foundation models such as SAM 2 collapse completely under extreme noise (12.1% mSAP at -20 dB) because their attention mechanisms rely on high-frequency texture cues. FPicker demonstrates that explicitly modeling the underlying physical manifold provides superior resilience in signal-starved regimes.
• Boundary Replicate Padding for Open Contours: Truncating gradient flows across logical endpoints via replicate padding in graph convolutions is a simple, effective, and reproducible solution for adapting active contours to open structures.
• Data-Dependent Global Anchoring: Max-pooling vertex features into a global descriptor dynamically broadcasts structural continuity cues from high-signal segments to noise-drowned vertices without imposing rigid coordinate embeddings.

Limitations & Future Work

• Absence of Branching Topology Support: FPicker assumes non-cyclic open curves and cannot represent branching structures (e.g., Y-junctions). While molecular branching is absent in target Cryo-EM filaments and apparent crossings are 2D projection overlaps, extending the framework with dynamic node degrees is required for vascular or neuronal arborization tracing.
• Tendency Toward Over-Merging: The strong continuity prior can occasionally bridge closely aligned independent filaments, resulting in an Over-Merging Rate (OMR) of 7.96%. Developing tip-repulsion mechanisms without compromising noise tolerance remains an open direction.

Related Work & Insights

• vs. crYOLO / YOLOv8: Box detectors suffer from ghost center drift when thin filaments are surrounded by noise; FPicker directly predicts a topological centroid and vector fields to anchor proposals on the protein backbone.
• vs. Topaz / SAM 2: Pixel-wise segmentation models merge overlapping filaments at 2D junctions and produce fractured skeletons under noise; FPicker preserves instance-level identity as deformable continuous curves.
• vs. Deep Snake / CurveGCN: Traditional closed active contours introduce cyclic tip shrinkage; FPicker enforces open-boundary conditions to maintain accurate tangent directions at filament extremities.

Rating

• Novelty: ⭐⭐⭐⭐⭐ Introduces the first open-curve graph evolution framework with center-endpoint representation, addressing long-standing ghost center and tip retraction degeneracies in Cryo-EM.
• Experimental Thoroughness: ⭐⭐⭐⭐⭐ Comprehensive evaluation across physical forward simulations (Cryo-Sim), real-world transfer (EMPIAR), few-shot data efficiency, and detailed component ablations.
• Writing Quality: ⭐⭐⭐⭐⭐ Exceptionally clear narrative, elegant mathematical formulations, and insightful problem framing.
• Value: ⭐⭐⭐⭐⭐ Provides a resilient geometric foundation for automating 3D helical reconstruction in Cryo-EM and demonstrates a robust paradigm for AI for Science under extreme low SNR.

Related Papers

• [AAAI 2026] Distributional Priors Guided Diffusion for Generating 3D Molecules in Low Data Regimes
• [ICML 2026] Protein Circuit Tracing via Cross-layer Transcoders
• [ICML 2026] Circuit Tracing in Autoregressive Protein Language Models
• [ICML 2026] Towards A Generative Protein Evolution Machine with DPLM-Evo
• [CVPR 2026] cryoSENSE: Compressive Sensing Enables High-throughput Microscopy with Sparse and Generative Priors on the Protein Cryo-EM Image Manifold