CVPR 2025 Earth Science Geochemical anomaly detection unsupervised learning Transformer benchmark dataset mineral exploration self-supervised pre-training

GeoChemAD: Benchmarking Unsupervised Geochemical Anomaly Detection for Mineral Exploration¶

Conference: CVPR 2025
arXiv: 2603.13068
Code: GitHub
Area: Self-Supervised
Keywords: Geochemical anomaly detection, unsupervised learning, Transformer, benchmark dataset, mineral exploration, self-supervised pre-training

TL;DR¶

This paper proposes the GeoChemAD open-source benchmark dataset (comprising 8 subsets covering multiple regions, sampling sources, and target elements) and the GeoChemFormer framework. By employing spatial context self-supervised pre-training and elemental dependency modeling, it achieves unsupervised geochemical anomaly detection and obtains state-of-the-art AUC across all subsets.

Background & Motivation¶

Importance of Geochemical Anomaly Detection (GAD): Geochemical anomalies refer to elemental concentrations that deviate from regional baselines. They serve as critical indicator signals for the presence of mineralization systems and are vital for mineral exploration.

Two Core Bottlenecks of Existing Research: - Data Secrecy: The vast majority of GAD studies utilize private datasets (mainly from the China Geological Survey), leading to non-reproducible results and preventing fair comparison among different methods. - Single Scenario: Most studies evaluate models only within a single region, a single sampling source (typically stream sediments), and a single target element (typically gold), leaving model generalization capabilities unclear.

Advantages and Limitations of Unsupervised Methods: Unsupervised methods do not require labeled mineralization points and exhibit better generalization. However, the core challenge is that the detected anomalies may be unrelated to the target mineralization elements.

Under-exploration of Transformers in GAD: Despite the outstanding performance of Transformers in multiple fields, their application in GAD lacks systematic research on self-supervised pre-training.

Method¶

GeoChemAD Dataset¶

Data Source: Public data from the Geological Survey of Western Australia (GSWA), compiled in the GDA2020 coordinate system.
8 Subsets: Covering three sampling sources: stream sediments (2), rock chips (3), and soils (3).
Diversified Target Elements: Au, Cu, W, Ni, breaking the limitations of previous studies that focused solely on gold.
Large Spatial Scale Span: Areas range from ~6 km² to ~8,500 km², with significant differences in sampling density.
Annotation: Each subset contains 7-32 known mineralization points as positive samples.

GeoChemFormer Framework (Two-Stage)¶

Stage 1: Spatial Context Learning (SCL)¶

Neighborhood Construction: For each query point \(p_i\), \(K\) nearest neighbors are retrieved using a KD-tree, where each neighbor is encoded as \(\mathbf{t}_j = [\Delta x_j, \Delta y_j, \mathbf{f}_j]\) (relative displacement + multi-element concentrations).
Encoder Input: \(\mathcal{S} = [\mathbf{e}, \mathbf{q}_i, \mathbf{t}_1, \ldots, \mathbf{t}_K]\), where \(\mathbf{e}\) represents the target element token, and \(\mathbf{q}_i\) is the query position token.
Self-Supervised Objective: Predicting the target element concentration at the query location from neighbor samples, optimized via the MSE loss \(\mathcal{L}_{sc}\).
Design Motivation: Force the model to learn spatial context from surrounding geochemical patterns, where the target element token aligns the learned representations with the specific target mineralization element.

Stage 2: Elemental Dependency Modeling + Anomaly Detection¶

Input Sequence: \(\mathcal{S}' = [W_g \mathbf{q}'_i, \mathbf{u}_1, \ldots, \mathbf{u}_c]\), consisting of the geo-context token and individual element tokens.
Element Token Design: \(\mathbf{u}_c = W_e[\text{Embed}(c) | W_v x_{i,c}]\), which combines elemental identity embedding and concentration values.
Transformer Encoder learns the inter-element dependency conditioned on the spatial context.
Anomaly Score: Mean Squared Reconstruction Error \(s_i = \frac{1}{C}\sum_{c=1}^{C}(x_{i,c} - \hat{x}_{i,c})^2\). Samples deviating from the learned elemental dependency patterns receive high anomaly scores.

Key Designs¶

Target Element-Aware Representation: Guided spatial context learning using target element tokens addresses the pain point that "detected anomalies may be unrelated to the target element."
Spatial-Compositional Decoupling: Learning spatial context in Stage 1 and element dependencies in Stage 2, decoupling these two types of information.

Key Experimental Results¶

Main Results (Average AUC of 8 Subsets)¶

Method Category	Representative Method	Average AUC	Variance
Statistical Methods	Z-score/MD/KNN	0.50-0.58	High
Classical ML	IF/OSVM	~0.58	Medium
AE	AutoEncoder	0.7046	0.0220
VAE-GAN	VAE-G	0.7279	0.0041
Vanilla Transformer	T1	0.7147	0.0031
GeoChemFormer	T2	0.7712	0.0039

Subset-level Highlights¶

soil3 (Ni): GeoChemFormer achieves an AUC of 0.8334, significantly outperforming the second-best VAE-CG (0.6509).
soil1 (Au): GeoChemFormer achieves an AUC of 0.8704, outperforming the second-best T1 (0.7242).
rock2 (Au): AE performs the best (0.9185), indicating that local geological settings dictate different optimal methods.

Data Preprocessing Ablation¶

Compositional Data Closure Handling: Isometric Log-Ratio (ILR) transformation performs the best on average (0.6788), followed by Centered Log-Ratio (CLR) (0.6771), while raw concentration performs the worst (0.6406).
Feature Selection: LLM-assisted selection is the best on average (0.7412) vs. manual selection (0.6419).
Neighborhood Size K: K=128 serves as a reasonable trade-off across datasets, though the optimal K varies by dataset.

SCL Pre-training Ablation¶

rock2 reaches its peak performance at 20 epochs (0.919), whereas soil2 requires 60 epochs (0.821).
The number of pre-training epochs needs to be adjusted based on the complexity of spatial distribution.

Highlights & Insights¶

First Open-source GAD Benchmark: Resolves the long-standing reliance on private datasets in this field, with 8 subsets covering diverse geological settings.
Target Element-Aware Unsupervised Detection: Addresses the core issue that "detected anomalies may be unrelated to target mineralization" via target element tokens.
Comprehensive Systematic Benchmark: Complete comparison spanning statistical methods -> classical ML -> deep generative learning -> Transformers.
In-depth Analysis of Data Preprocessing: Ablation on compositional transformations, feature selection, and interpolation methods provides solid guidelines for practitioners.
Discovery of LLM-assisted Feature Selection: LLMs can effectively select geochemical element combinations relevant to mineralization.

Limitations & Future Work¶

Weak Connection to Mainstream CV/ML: Although accepted at CVPR, the task is highly domain-specific, and the transferability of the method remains to be verified.
Low Absolute AUC Values: The average AUC of 0.77 falls short of the practical standards required for real-world mineral exploration.
Extremely Scarce Positive Samples (7-32), which limits evaluation stability (despite averaging over 20 random sampling trials).
Questionable Generalizability of the Method: AE outperforms GeoChemFormer on rock2 (0.9185 vs. 0.81), indicating a lack of consistent superiority.
Lack of In-depth Failure Case Analysis for GeoChemFormer: Underperforms AE on rock2, rock3, and sed2.
Missing Computational Overhead Comparison: The efficiency of KD-tree neighborhood search paired with Transformer encoding requires further investigation.

Rating¶

Novelty: ⭐⭐⭐ Methodological innovations are somewhat limited (standard Transformer + self-supervision), with the dataset contribution being more prominent.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extremely comprehensive benchmark and ablation analyses.
Writing Quality: ⭐⭐⭐⭐ Well-organized with detailed dataset descriptions.
Value: ⭐⭐⭐ Holds benchmarking value for the geochemistry community, though methodological inspiration for the CV community is limited.