Skip to content

title: >- [Paper Note] Rosetum3D: A Large-Scale 3D Vision Dataset from Preharvest Roses description: >- [ECCV 2026][3D Vision][3D Vision Dataset] Introduces Rosetum3D, a large-scale RGB-D dataset of 21,114 frames and 46,848 preharvest roses with botanically defined 2D-to-3D keypoint annotations and multi-task benchmarks. tags: - ECCV 2026 - 3D Vision - 3D Vision Dataset - Preharvest Rose Localization - Keypoint Estimation - Agricultural Vision date: 2026-09-19 content_hash: f00c847132babaf6

Rosetum3D: A Large-Scale 3D Vision Dataset from Preharvest Roses

Conference: ECCV 2026
Paper: ECCV Official
Code: https://huggingface.co/datasets/WaterMelon2333/Rosetum3D/tree/main
Area: 3D Vision
Keywords: 3D vision dataset, preharvest rose localization, keypoint estimation, depth backprojection, agricultural vision

TL;DR

Addressing the acute lack of 3D benchmarks in floriculture automation, this paper introduces Rosetum3D, comprising 21,114 high-resolution RGB-D images and 46,848 annotated rose instances across commercial greenhouses, establishing a botanically guided 2D-to-3D backprojection labeling paradigm and multi-task benchmarks spanning detection, pose estimation, and temporal 3D localization.

Background & Motivation

The global cut-rose industry continues to expand rapidly, driven by escalating consumer demand for commercial and ornamental floriculture. Evaluating preharvest quality relies on rigorous multidimensional geometric criteria, including petal count, corolla opening geometry, stem length, and overall vertical erectness. Historically, these phenotype assessments have been conducted manually by human experts after harvesting. This post-harvest grading workflow is not only time-consuming and labor-intensive, but destructive evaluation cannot provide proactive feedback for adaptive climate control, fertilization scheduling, or selective robotic harvesting during cultivation. Applying 3D computer vision technologies directly inside operational greenhouses presents a compelling pathway toward non-destructive automated phenotyping. However, existing flower datasets are predominantly restricted to 2D species classification or planar bounding box detection, failing to provide spatial coordinates and continuous stem geometry in 3D physical space.

General 3D vision datasets developed for autonomous driving (e.g., KITTI, nuScenes) or indoor room understanding (e.g., ScanNet, SUN RGB-D) cannot be readily adapted to greenhouse environments. In production greenhouses, crops grow in dense, highly clustered arrangements with severe inter-plant occlusions, delicate and non-rigid petal morphologies, and extreme lighting variations caused by sunlight and artificial grow lamps. Under such challenging conditions, standard LiDAR scanning or multi-view geometric reconstruction (such as Structure-from-Motion and Neural Radiance Fields) suffer from frequent point cloud dropouts, boundary noise, and ghosting artifacts, rendering direct 3D mesh or point cloud semantic annotation practically unfeasible.

The angle of attack in this paper is to bypass error-prone global 3D geometric reconstruction in favor of combining mature, high-precision 2D keypoint annotation with calibrated depth sensing. Core idea: by collecting multi-view RGB-D sequences with a chest-mounted structured-light sensor along greenhouse plant rows, the authors establish a botanically defined 2D-to-3D backprojection annotation framework combining 5 anatomical keypoints and 4 spatial interpolation points, construct the first large-scale preharvest rose 3D vision benchmark, and propose a cross-frame Re-ID temporal consistency optimization scheme for spatial localization.

Method

Overall Architecture

The construction of Rosetum3D and its downstream localization baselines encompasses four interconnected phases: systematic greenhouse RGB-D data acquisition with precise sensor calibration, botanically defined 2D-to-3D annotation with physical scale validation, comprehensive multi-task agricultural benchmark evaluation, and a temporal consistency optimization framework for 3D localization.

The overall operational workflow proceeds hierarchically: operators equipped with chest-mounted active-passive stereo sensors record continuous RGB-D streams while traversing plant rows; human annotators label 2D bounding boxes and 9 skeleton keypoints on RGB frames, which are then backprojected into 3D camera coordinates via calibrated intrinsic matrices and synchronized depth maps; during downstream 3D spatial inference, 2D keypoint predictions and monocular depth estimates are merged and subsequently refined across adjacent video frames using visual instance re-identification and temporal linear interpolation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Continuous Greenhouse Capture<br/>Chest-mounted RGB-D (30Hz, 1280x800)"] --> B["Multi-View Structured-Light Calibration<br/>Checkerboard Intrinsic/Extrinsic Alignment"]
    B --> C["Botanical 2D-to-3D Hybrid Annotation<br/>5 Anatomical Keypoints + 4 Inter-Points"]
    C --> D["Depth Backprojection Physical Validation<br/>Metric Tape Verification (0.14 cm MAE)"]
    D --> E["Multi-Task Agricultural Benchmark Suite<br/>Detection/Pose/Depth/Matching/Re-ID"]
    E --> F["Cross-Frame Re-ID & Temporal Smoothing<br/>Instance Association & Prior Fusion"]
    F --> G["Refined 3D Rose Localization<br/>Temporally Consistent 3D Keypoints"]

Key Designs

1. Multi-View Structured-Light Calibration: Mitigating Greenhouse Optical Noise and Depth Distortion Greenhouse environments exhibit harsh ambient lighting fluctuations, direct solar illumination, and complex plant leaf reflections that frequently degrade the signal-to-noise ratio of conventional active infrared time-of-flight or monocular depth sensors. The authors utilize Orbbec Gemini 335L/336L stereo vision cameras, which integrate active structured-light illumination with passive binocular stereo disparity computation, ensuring reliable depth capture across diverse lighting conditions. The camera is strapped to the operator's chest and tethered to an Android mobile device, recording synchronized RGB and depth streams at 30 Hz at a resolution of 1280×800. Operators maintain a steady walking speed of 0.3–0.5 m/s along plant rows, preserving a 0.8–1.2 m distance with a fixed downward tilt angle to capture both floral heads and lower root stalks. Pre-collection calibration utilizes planar checkerboards across multiple distances following the ScanNet protocol to accurately solve for color/depth sensor intrinsics \(\mathbf{K}\) and extrinsic alignment matrices.

2. Botanical 2D-to-3D Hybrid Annotation: Decoupling Complex Occlusion from 3D Point Cloud Reconstruction To resolve the bottleneck where dense crop foliage prevents reliable manual labeling directly in 3D meshes, the authors formulate a fine-grained anatomical keypoint labeling framework inspired by human pose estimation. Annotators first delineate a 2D bounding box encompassing each rose specimen and then mark 5 biologically meaningful landmarks: - Corolla Center: The geometric centroid of the flower corolla, denoting the visual floral center; - Sepal-Pedicel Junction: The anatomical intersection where the basal sepal whorl converges onto the pedicel; - Stem-Pedicel Transition: The morphological transition locus where the main stem diameter undergoes significant reduction into the floral stalk; - Median Point: The spatial midpoint along the vertical axis of the above-ground shoot structure; - Soil-Emergence Point: The crown interface where the primary stalk emerges from the soil.

Because stems exhibit continuous natural curvature, 5 sparse points cannot capture structural inflection. Four auxiliary interpolation points are added: one between the sepal junction and stem transition, one between the stem transition and median point, and two dividing the median-to-soil segment into thirds, producing a 9-keypoint continuous representation. Each 2D coordinate \((u, v)\) is backprojected into 3D camera coordinates using the calibrated camera intrinsic matrix \(\mathbf{K}\) and the synchronized depth map \(D(u, v)\): $$ \mathbf{P}_{3D} = \mathbf{K}^{-1} \cdot \begin{bmatrix} u \ v \ 1 \end{bmatrix} \cdot D(u, v) $$ Heavily occluded points are flagged with an invisible attribute and omitted from metric accuracy evaluations. To empirically confirm annotation precision, physical tape measures were installed alongside primary stems across 137 validation trials, demonstrating that depth-backprojected 10 cm stem segments yielded a mean absolute error (MAE) of only 0.14 cm.

3. Multi-Task Agricultural Benchmark Suite: Establishing Comprehensive Preharvest Perception Baselines Rosetum3D establishes standardized benchmarks across six primary visual tasks: 1. 2D Rose Object Detection: Evaluating Faster R-CNN, DETR, RT-DETR, and YOLOv5 under cluttered canopy foliage; 2. 2D Rose Keypoint Localization: Adapting MMPose architectures, benchmarking Top-down models (RTMPose, ViTPose) and Bottom-up models (YOLOXPose, DEKR+HRNet) on flexible, slender stems; 3. Monocular Depth Estimation: Evaluating models in zero-shot inference (trained on NYU Depth V2) versus in-domain fine-tuning splits (Depth Anything, BinsFormer, DepthFormer); 4. Local Feature Matching: Creating Rosetum3D-1800 with COLMAP pseudo-ground-truth pairs to benchmark zero-shot keypoint matchers (DeDoDe, DKM, RoMa) under high visual self-similarity; 5. Instance Re-Identification: Tracking 1,923 individual plants across 36,712 images to evaluate deep person Re-ID architectures (OSNet, TransReID, SOLIDER) on subtle phenotypic variations; 6. 3D Rose Keypoint Localization: Integrating 2D pose inference with depth estimations to evaluate metric spatial joint errors.

4. Cross-Frame Re-ID and Temporal Consistency Optimization: Suppressing Single-Frame Depth Jitter Single-frame 3D keypoint inference often suffers from erratic positional jitter and outlier spikes caused by leaf edge ambiguity, specular reflections, or localized depth sensor dropouts. The authors introduce a multi-frame temporal optimization framework over three consecutive frames (\(t-1, t, t+1\)). First, a pre-trained OSNet Re-ID model associates corresponding rose instances across adjacent frames using visual appearance embeddings. Next, a temporal prior position for the center frame is computed via linear interpolation: $$ \mathbf{P}{\text{interp}}^t = \frac{1}{2} \left( \mathbf{P}} + \mathbf{P{t+1} \right) $$ The final optimized 3D keypoint position \(\mathbf{P}_t^*\) is calculated by balancing the instantaneous observation against the temporal prior, where the observation weight \(w_d = W_{\text{depth}} \cdot \text{conf}\) is modulated by the 2D keypoint confidence score, and the prior weight \(w_t = W_{\text{temp}}\) is held constant: $$ \mathbf{P}_t^* = \frac{w_d \mathbf{P}_t + w_t \mathbf{P} $$ This temporal smoothing formulation effectively dampens high-frequency depth discontinuities and stabilizes spatial coordinate trajectories.}}^t}{w_d + w_t

A Worked Example

Consider an individual partially occluded rose captured in a sequence: 1. Detection and 2D Keypoint Regression: At frame \(t\), the YOLOv5 detector extracts the rose bounding box, and the ViTPose top-down head predicts the corolla center at \((u_t, v_t) = (640.5, 312.4)\) with keypoint confidence \(\text{conf}_t = 0.92\); 2. Depth Query and Backprojection: Querying the fine-tuned BinsFormer depth map yields \(D(u_t, v_t) = 0.985\text{ m}\). Applying inverse perspective projection via camera matrix \(\mathbf{K}\) produces the raw instantaneous 3D camera coordinate \(\mathbf{P}_t = [0.035, -0.120, 0.985]^T\text{ m}\); 3. Cross-Frame Re-ID Association: OSNet matches the corresponding rose instance in adjacent frames \(t-1\) and \(t+1\), yielding coordinates \(\mathbf{P}_{t-1} = [0.065, -0.121, 0.978]^T\text{ m}\) and \(\mathbf{P}_{t+1} = [0.004, -0.119, 0.990]^T\text{ m}\). Linear interpolation produces temporal prior \(\mathbf{P}_{\text{interp}}^t = [0.0345, -0.120, 0.984]^T\text{ m}\); 4. Weighted Fusion Update: With hyperparameters \(W_{\text{depth}}=1.0\) and \(W_{\text{temp}}=0.5\), weights evaluate to \(w_d = 0.92\) and \(w_t = 0.5\). Normalized fusion computes \(\mathbf{P}_t^* = [0.0348, -0.120, 0.9846]^T\text{ m}\), filtering out transient depth noise and stabilizing the 3D position.

Key Experimental Results

Main Results

The authors conduct comprehensive experiments benchmarking 2D detection, 2D keypoint estimation, monocular depth estimation, and 3D keypoint localization.

Table 1: 2D Rose Object Detection Baselines on Rosetum3D | Algorithm | Backbone | \(\text{AP}\) | \(\text{AP}_{50}\) | \(\text{AP}_{75}\) | \(\text{AP}_M\) | \(\text{AP}_L\) | | :--- | :--- | :---: | :---: | :---: | :---: | :---: | | Faster R-CNN | ResNet50 | 31.0 | 66.8 | 24.7 | 13.1 | 32.1 | | DETR | ResNet50 | 33.0 | 66.5 | 28.8 | 14.2 | 34.3 | | RT-DETR | ResNet50 | 33.6 | 64.8 | 30.7 | 13.8 | 34.7 | | YOLOv5 | CSPDarkNet53 | 35.0 | 67.1 | 32.4 | 19.1 | 35.9 |

Note: \(\text{AP}_M\) evaluates medium objects (\(32\times 32 \sim 96\times 96\) pixels), and \(\text{AP}_L\) evaluates large objects (\(>96\times 96\) pixels).

Table 2: 2D Rose Keypoint Localization Results | Pipeline Type | Method | Backbone | Detector | \(\text{AP}\) | \(\text{AP}_{50}\) | \(\text{AP}_{75}\) | \(\text{AR}\) | \(\text{AR}_{50}\) | \(\text{AR}_{75}\) | | :--- | :--- | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | Top-down | RTMPose | CSPNeXt-L | YOLOv5 | 16.6 | 45.2 | 8.6 | 27.1 | 54.9 | 22.9 | | Top-down | RTMPose | CSPNeXt-L | Ground Truth | 37.2 | 82.2 | 28.9 | 48.0 | 86.5 | 44.4 | | Top-down | ViTPose | ViT-L | YOLOv5 | 21.9 | 48.5 | 17.2 | 32.0 | 56.9 | 31.2 | | Top-down | ViTPose | ViT-L | Ground Truth | 42.1 | 85.8 | 37.2 | 53.6 | 90.7 | 53.7 | | Bottom-up | YOLOXPose | CSPDarknet | — | 25.0 | 60.3 | 19.3 | 53.7 | 88.7 | 55.2 | | Bottom-up | DEKR + HRNet | HRNet | — | 24.9 | 57.8 | 18.6 | 43.9 | 81.2 | 41.8 |

Ablation Study

Table 3: 3D Rose Localization under Different Depth Inferencer Sources (MPJPE in mm) | 2D Keypoint Inferencer | Detector | Depth Inferencer Source | Depth Encoder | \(\text{MPJPE} \downarrow\) | | :--- | :--- | :--- | :--- | :---: | | ViTPose | DETR | Depth Anything (Zero-shot) | ViT-L | 656.9 | | ViTPose | DETR | BinsFormer (Fine-tuned) | Swin-Large | 647.2 | | ViTPose | DETR | Ground Truth Depth (Sensor) | — | 556.8 | | RTMPose | DETR | Depth Anything (Zero-shot) | ViT-L | 650.6 | | RTMPose | DETR | BinsFormer (Fine-tuned) | Swin-Large | 644.5 | | RTMPose | DETR | Ground Truth Depth (Sensor) | — | 558.4 | | DEKR + HRNet | — | Depth Anything (Zero-shot) | ViT-L | 669.4 | | DEKR + HRNet | — | BinsFormer (Fine-tuned) | Swin-Large | 654.3 | | DEKR + HRNet | — | Ground Truth Depth (Sensor) | — | 522.8 | | YOLOXPose | — | Depth Anything (Zero-shot) | ViT-L | 653.2 | | YOLOXPose | — | BinsFormer (Fine-tuned) | Swin-Large | 645.1 | | YOLOXPose | — | Ground Truth Depth (Sensor) | — | 589.3 |

Table 4: Impact of Multi-Frame Temporal Consistency Framework (MPJPE in mm) | 2D Keypoint Inferencer | Detector | Multi-Frame Temporal Optimization | \(\text{MPJPE} \downarrow\) | Improvement | | :--- | :--- | :---: | :---: | :---: | | ViTPose | DETR | ✗ | 556.8 | Baseline | | ViTPose | DETR | ✓ | 539.1 | -17.7 mm | | RTMPose | DETR | ✗ | 558.4 | Baseline | | RTMPose | DETR | ✓ | 542.2 | -16.2 mm |

Key Findings

  • Detector Bottlenecks Hinder Top-Down Pose Estimation: When driven by predicted bounding boxes from YOLOv5, top-down approaches (ViTPose achieving 21.9 AP) lag behind bottom-up approaches (YOLOXPose at 25.0 AP). However, substituting detector boxes with ground-truth bounding boxes propels ViTPose performance to 42.1 AP, revealing that miss-detections and box boundary drift in dense crops are primary error sources.
  • In-Domain Fine-Tuning Essential for Agricultural Depth Estimation: Zero-shot depth foundation models trained on indoor datasets fail in greenhouses (Depth Anything yields an AbsRel of 0.498 and RMSE of 1.092 m). Fine-tuning BinsFormer on Rosetum3D drastically reduces AbsRel to 0.165 and RMSE to 0.734 m, proving the necessity of dedicated agricultural depth supervision.
  • Multi-Frame Interpolation Dampens Sensor Noise: Activating temporal consistency optimization reduces 3D joint positioning error (MPJPE) by 17.7 mm for ViTPose and 16.2 mm for RTMPose, corroborating the benefit of leveraging inter-frame visual continuity to suppress depth artifacts.

Highlights & Insights

  • Botanical Anatomical Decoupling for High-Fidelity 2D-to-3D Labeling: Adapting human kinematic skeleton conventions to plants (5 biological landmarks plus 4 geometric interpolation points) cleverly bypasses the intractable errors of multi-view 3D meshing, validated by a physical calibration tape MAE of only 0.14 cm.
  • Holistic Multi-Task Agricultural Benchmark Suite: Rosetum3D uniquely provides comprehensive benchmarking across 6 core tasks (2D detection, pose, monocular depth, local feature matching with Rosetum3D-1800, plant Re-ID across 1,923 specimens, and 3D localization), creating a foundational sandbox for agricultural embodied AI.
  • Deployable Lightweight Cross-Frame Smoothing: The combination of lightweight Re-ID association and temporal interpolation provides a computationally efficient smoothing strategy that can be deployed onto harvesting robotic arms or field inspection rovers without requiring heavy neural SLAM backends.

Limitations & Future Work

  • High Absolute Metric 3D Localization Error: The authors candidly acknowledge that overall MPJPE errors remain between 520 mm and 670 mm, which is too coarse for sub-centimeter robotic cutting shears, primarily caused by distant depth dropouts and extreme foliage occlusion.
  • Camera Ego-Motion Unmodeled in Temporal Smoothing: The multi-frame linear interpolation operates purely within camera coordinates under the idealized assumption of constant forward translation, omitting high-frequency orientation wobble from the human-borne camera mount.
  • Future Directions: Subsequent work should incorporate visual-inertial SLAM to recover precise 6-DoF camera poses for global bundle adjustment, and explore 3D Gaussian Splatting to achieve millimeter-level implicit reconstruction of fine plant stems.
  • vs ScanNet / SUN RGB-D (Indoor 3D Datasets): Conventional datasets focus on static rigid architectural entities where annotations can be directly painted onto 3D point clouds. Rosetum3D addresses non-rigid biological organisms under heavy occlusion where point clouds fracture, pioneering a robust 2D-to-3D projection workflow.
  • vs Pheno4D / Plant3D / ROSE-X (Agricultural 3D Datasets): Existing plant scanning benchmarks primarily capture potted plants or laser-scanned specimens in laboratory conditions (e.g., ROSE-X contains only 11 CT-scanned rosebushes). Rosetum3D delivers over 21k operational greenhouse images and 46k plant instances under real-world commercial production.
  • vs Human Pose Estimation Frameworks: By successfully adapting MMPose paradigms to floriculture, the paper demonstrates that plant stems and branching topologies can be treated as articulated biological kinematic trees, offering fresh methodologies for automated phenotyping.

Rating

  • Novelty: ⭐⭐⭐⭐☆ (Constructs the first large-scale 3D vision dataset for preharvest cut roses with a dedicated botanical 2D-to-3D keypoint workflow)
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ (Exhaustively benchmarks 6 distinct vision tasks across dozens of modern SOTA backbones)
  • Writing Quality: ⭐⭐⭐⭐☆ (Clear motivation, rigorous data statistics, and systematically structured benchmark comparisons)
  • Value: ⭐⭐⭐⭐⭐ (Fills a long-standing data void in protected floriculture, providing critical infrastructure for agricultural robotics and digital phenotyping)