SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning

Conference: ECCV2026
Paper: ECCV Official Page
PDF: Full Paper
Area: Multimodal Reasoning (vlm_reasoning)
Keywords: multimodal mathematical reasoning, visual-textual co-evolution, difficulty alignment, self-improvement, GRPO

TL;DR

SyncLoop jointly generates more complex mathematical images and questions, then selects training examples using the current model's answer error rate, improving Qwen2-VL-7B Geo-Sub-Aux accuracy from 40.4% to 52.4% after three evolution rounds (Table 2, p. 10).

Background & Motivation

For a vision-language model (VLM), solving geometry problems requires more than producing a long derivation: the derivation must refer to the correct points, edges, and relations in the diagram. Expanding only the question can introduce references to auxiliary lines absent from the image; adding only visual elements can leave the training objective at the level of simple recognition. Visual complexity and reasoning difficulty are therefore not two independent attributes that can simply be increased separately. The paper discusses this issue alongside OpenVLThinker's iterative training and visual data generation in R-CoT and MAVIS, but SyncLoop focuses on changing both modalities together.

Even a difficult, visually consistent question is not necessarily useful for the current model. Fixed data can become too easy as the model improves, while exclusively selecting completely unsolved problems can leave training with few usable successful trajectories. SyncLoop therefore addresses two interfaces: generated questions must use the new visual structure, and training selection must be reassessed against current model capability. Here, self-improvement does not mean eliminating external supervision, because question generation, answer generation, and quality filtering still depend on GPT-4o.

Core Idea: co-evolve images and questions in a data loop, then feed model errors back into training selection so that subsequent learning content changes with model capability.

Method

Overall Architecture

The inputs are a pretrained multimodal model and image-question-answer triplets, with Geometry3k supplying the seed data for geometry experiments. The data loop performs Visual Action and Reasoning Generation, Constraint-Guided Question Composition, and Consistency Validation; the model loop selects examples and learns through Error-Driven Model Updates. The outputs include both an updated model and evolved visual-textual data that can be processed again, rather than merely additional inference-time computation for a single question. The diagram follows Figure 2 and Algorithm 1; the update node describes the conceptual procedure, while whether SFT should recur in later rounds requires reading the ablations.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    A["Seed Problems and Model"] --> B["Visual Action and<br/>Reasoning Generation"]
    B --> C["Constraint-Guided<br/>Question Composition"]
    C --> D["Consistency Validation"]
    D --> E["Error-Driven<br/>Model Updates"]
    E --> F["Updated Model and Evolved Data"]
    F -->|Next data evolution round| B
    F -->|Update difficulty evaluator| E

Key Designs

1. Visual Action and Reasoning Generation: give the new image mathematical structure for question generation

For geometric diagrams, the system first extracts key-point coordinates using OCR and vision-based methods, establishing spatial references for subsequent drawing. GPT-4o uses these coordinates and the original image to decide whether auxiliary structures are necessary and produces actions, Python code, and a natural-language reasoning trace. Executing the code produces an image with connecting, parallel, or perpendicular lines, rather than merely claiming in text that a line exists. The coordinate representation reduces inconsistencies in locating diagram edits; it does not replace all geometric semantic understanding. The retained reasoning trace also tells the next stage which theorems and intermediate conditions in the original solution could become new questions.

Function tasks follow a related approach: GPT-4o decides whether plotting the function would assist the solution before continuing the visual-textual evolution process. Page 8 explicitly limits the subsequent formal image description to geometric diagrams, so these details should not be indiscriminately attributed to every function problem. During data generation, an auxiliary line may initially assist a solution, but new training questions are constructed around the added structure. This is therefore not equivalent to adding a helpful line to the original question at test time; it changes the conditions and reasoning space available for the next generation of questions.

2. Constraint-Guided Question Composition: turn added geometric relations into multistep tasks

GPT-4o converts the evolved diagram into a formal description of points, lines, shapes, and positional relations. Question generation uses this description and the preceding reasoning trace under three principles: mathematical constraints, new theorems and concepts, and backward reasoning. The constraint principle extracts perpendicularity, equality, or sub-image relations; the theorem principle connects those conditions to relevant knowledge such as the Pythagorean theorem. Backward reasoning reuses intermediate conclusions or theorems from the original solution while reorganizing the givens and the requested quantity. The principles provide complementary sources of questions, not three separately trained models.

Each image typically yields 4 to 10 sub-problems, depending on its complexity. Following a role-playing strategy from R1-OneVision, the system examines similarities and differences among these sub-problems and associates them with corresponding sub-image elements. It then composes questions using thematic and contextual connections, for example making a solution depend on relations in two different parts of the diagram. The intended increase is in reasoning dependencies across conditions, not merely question length. Figure 3 uses increasing reasoning length across rounds as supporting evidence of difficulty growth, but capability alignment relies on answer errors rather than direct token-count filtering.

3. Consistency Validation: establish sample credibility before measuring difficulty

GPT-4o solves each new question with its formal description three times, producing a reasoning trace and final answer on each attempt. Only samples with three matching final answers are retained; a further model-based check removes questions that violate the original image constraints. The first check targets answer stability, while the second targets compatibility between the question and visual content. Both concern data quality rather than student-model difficulty and must not be confused with the later 32-generation assessment.

Three matching answers do not constitute a mathematical proof: one teacher can repeat the same mistake. If a formal description omits a visual condition, repeated solutions may also remain consistent under an incorrect premise. These filters can reduce obvious noise, but their reliability remains limited by the teacher and image description quality. The paper does not establish that every generated example is verified by an independent symbolic solver.

4. Error-Driven Model Updates: use current capability to choose the next training content

The current model generates 32 answers for each quality-filtered question, with the fraction of incorrect attempts defining difficulty. The following equation restates the textual definition in Section 3.3; it is not an additional loss function:

\[ e(I,Q,G)=\frac{1}{32}\sum_{j=1}^{32}\mathbf{1}[\hat A_j\ne G]. \]

Here, \(G\) is the reference answer and \(\hat A_j\) is the answer from generation \(j\) after processing under the answer-checking rule. Section 3.3 specifies retaining \(e>0.3\), whereas Section 4.2 describes the final setting as \(e\geq0.3\). Because the error rate is an integer multiple of \(1/32\), both descriptions require at least 10 incorrect attempts with 32 generations. This is not a narrow band around 0.3, and the default rule does not explicitly exclude questions answered incorrectly on all 32 attempts. Consequently, suitable difficulty should be understood as an empirical thresholding heuristic, not a strict guarantee that every example lies near the model's capability boundary.

Selected questions drive model updates, and the updated model evaluates subsequent data, completing the feedback loop. The paper describes SFT as establishing output structure and GRPO as improving answer quality through rewards, but their scheduling has an important ablation caveat. Table 3 reports second-round Geo-Sub-Aux accuracy of 40.7% when repeating SFT plus GRPO from the previous model, versus 48.0% when continuing GRPO alone. The third-round values are 42.9% and 52.4%, respectively, so the main-table gains cannot simply be attributed to repeating SFT in every round. Algorithm 1's general procedure must be distinguished from the strongest ablation setting; the main text alone does not unambiguously reconstruct every checkpoint's training schedule.

A Worked Example

Figure 2 uses a circle geometry problem with secant \(AEB\) and tangent \(AD\) from an external point \(A\), asking for a proof of \(AD^2=AE\cdot AB\). The visual-action stage proposes connecting \(DE\) and \(DB\) and actually draws these lines through code execution. The reasoning trace uses angle relations and similar triangles to establish the tangent-secant relation, supplying reusable intermediate concepts for question generation. The formal description records the added segments and relations such as perpendicularity between the tangent and radius, after which constraint-guided composition combines sub-problems. A new question enters the student's 32-generation assessment only after passing three-answer consistency and image-constraint checks. If its error count reaches the threshold, it can enter training; otherwise, that round reserves the learning budget for more challenging examples. Figure 2 does not provide a complete new question and sampling-error count for this exact example, so no purportedly verified final sample is invented here.

Loss & Training

SFT uses the unified output format <think></think><answer></answer> to provide a stable structure for subsequent reward checks. Equation (1) trains the reasoning output with conditional negative log-likelihood; with its parentheses normalized, it reads:

\[ \mathcal{L}_{\mathrm{SFT}}=-\mathbb{E}_{(I,Q,R)\sim D}[\log\pi_\theta(R\mid I,Q)]. \]

Here, \(R\) denotes the supervised output trajectory; the objective is to organize reasoning and answers for subsequent RL, not to introduce a new visual encoder. GRPO uses final-answer correctness and format compliance rewards, with regular expressions extracting answers for comparison against the reference. The paper also requires reasoning steps to be sequentially organized and penalizes ordering violations, but this is not equivalent to verifying each mathematical step. Equation (2) is corrupted in the cached extraction, so the complete GRPO objective and individual reward weights cannot be reliably recovered; a standard formula is not substituted as if it were the source.

SFT uses LLaMA-Factory with batch size 128 and learning rate \(10^{-5}\); GRPO uses VLM-R1. GRPO uses 32 rollouts per question in the first round and 8 in later rounds, temperature 0.9, and KL coefficient \(\beta=0\). Training rollouts and the 32 generations used for difficulty measurement serve different purposes: reducing later training rollouts to 8 does not imply an 8-generation filtering rule. Experiments use 32 NVIDIA V100-32GB GPUs, so a small selected training set should not be equated with low total computational cost.

Key Experimental Results

Main Results

Geometry3k contains 3,002 problems, split into 2,101 training, 300 validation, and 601 test examples. The authors select 274 test images requiring auxiliary lines: evaluation on the original diagrams is Geo-Sub, while evaluation with corresponding added lines is Geo-Sub-Aux. These are not independently collected datasets, and the difference between their columns is not itself a training gain. The following selection from Table 2 (p. 10) reports accuracy in %; data amounts are those reported for each setting, not total generation costs.

Model / Round	Reported Training Data	Geo-Sub	Geo-Sub-Aux	MathVista ALL
Qwen2-VL-2B	Not listed	28.0	29.1	43.0†
SyncLoop-2B, round 3	0.4k	38.2	42.2	50.2
Qwen2-VL-7B	Not listed	40.4	40.4	60.0
SyncLoop-7B, round 1	0.6k	45.5	46.9	62.1
SyncLoop-7B, round 2	0.6k	46.9	48.0	62.4
SyncLoop-7B, round 3	0.4k	50.9	52.4	63.2

† Table 2 marks some values as taken from earlier papers; the 2B MathVista baseline carries that mark and should not be described as uniformly reproduced under identical conditions. The 7B model gains 12.0 percentage points on Geo-Sub-Aux and 10.5 points on original-image Geo-Sub, indicating that gains are not limited to auxiliary-line test inputs. MathVista ALL increases from 60.0% to 63.2%, a 3.2-point gain that is smaller than the improvement on the specialized geometry subset. For 2B, Geo-Sub-Aux rises from 29.1% to 42.2%, a 13.1-point gain, although models of different sizes should not be ranked solely by their improvement margins.

Ablation Study

Table 5 (p. 12) compares second-round error-rate filtering; values before and after each slash refer to current-round data \(D_t\) and data including history \(D_{1\sim t}\), respectively. These data regimes must be compared separately rather than combining their best entries into one training configuration.

Selection Setting	Data: Current / Cumulative	Geo-Sub-Aux: Current / Cumulative (%)
Fullset	0.83k / 1.48k	48.4 / 46.6
Error-rate threshold 0.3	0.48k / 0.64k	49.1 / 48.0
Error-rate threshold 0.6	0.44k / 0.54k	50.9 / 47.3
Error-rate threshold 0.9	0.38k / 0.44k	48.4 / 47.3
Error-rate setting 1.0	0.34k / 0.35k	46.2 / 46.5

With current-round data alone, threshold 0.6 reaches 50.9%, above 49.1% at 0.3; with historical data included, 0.3 performs better at 48.0%. Table 5 therefore supports a joint effect of difficulty threshold and data mixing, not the claim that 0.3 is optimal under every condition. The authors choose 0.3 and refer to additional generalization experiments in the appendix; that appendix is absent from the available cache and cannot be further checked here.

Table 8 (p. 13) compares original images paired with complex text against complex images paired with complex text; Section 4.2's context identifies these as 7B Geo-Sub-Aux results, although the table header does not separately name the metric.

Round	Original Images + Complex Text (%)	Complex Images + Complex Text (%)
Round 1	47.27	46.9
Round 2	45.82	48.0
Round 3	51.27	52.4

Joint evolution performs better in rounds 2 and 3, but not round 1; this ablation does not establish that increasing image complexity helps in every round.

Key Findings

• Table 10 (p. 14) reports second-round single-principle results of 47.2% for constraints, 46.9% for new theorems, and 47.3% for backward reasoning, versus 48.0% jointly; different single-principle data amounts prevent fully isolating the principles themselves.
• Table 11 (p. 14) reports 46.6% in round 2 without reasoning data, below the main table's 48.0%; retaining reasoning traces helps but does not explain all gains.
• Table 6 (p. 13) reports 72.0% / 71.0% on FunctionQA / Function-Plot with added function data in round 2, versus 58.0% / 58.0% for the baseline; this separate 0.7k-data setting should not be merged into the geometry main table.

Highlights & Insights

• Data generation and selection solve different problems: generation preserves visual-textual consistency, while selection exposes the model to content it has not mastered. Separating the filters makes label noise and difficulty mismatch easier to distinguish.
• New visual structure becomes meaningful data evolution only when a new question actually uses it. This could inform other structured-chart tasks, but direct transfer to natural-image scenes is not evaluated here.
• Table 5 shows that difficulty is not a fixed label independent of the model or historical data. Curriculum strategies should be designed together with data mixing rather than by tuning a threshold in isolation.

Limitations & Future Work

• External teacher dependence is substantial: GPT-4o generates questions and answers and participates in filtering, so three-way agreement can retain correlated errors. Independent geometric or algebraic verification is a possible extension, not an implemented result here.
• The default threshold has no explicit upper bound and can retain unsolved or mislabeled examples. Degradation at the most extreme error-rate setting in Table 5 motivates stronger learnability filtering.
• Small data volume does not make the full process cheap: teacher calls, image generation, 32-generation difficulty assessments, and training on 32 GPUs all contribute. The main text does not provide a complete end-to-end cost accounting.
• The training description contains a tension: Algorithm 1 applies SFT before GRPO every round, whereas Table 3 supports continuing GRPO directly in later rounds. Reproduction requires confirming checkpoint-specific schedules rather than following the algorithm box alone.
• The available cache contains the main paper and references but not the cited appendix, and Equation (2) is unreadable in full. No MathVerse numbers, complete prompts, or unverified reward details are filled in from those missing materials.

Related Work & Insights

• vs OpenVLThinker: both study iterative self-improvement; SyncLoop adds joint visual-textual generation and model-error filtering. Table 9 also compares different data volumes, including 5k and 0.6k, and does not establish equal total cost.
• vs R-CoT / MAVIS: the paper positions these as related visual-data evolution approaches; SyncLoop emphasizes changing text alongside added visual conditions and connecting the resulting data to model feedback.
• vs STaR / ReST: all involve generation, filtering, and retraining. SyncLoop additionally changes the visual input and question, rather than only selecting better responses to fixed inputs.
• vs MM-EUREKA: the paper describes this related approach as removing trivial and unsolvable examples; SyncLoop's default lower-bound threshold does not strictly remove all-wrong questions, so their filters are not interchangeable.

Rating

• Novelty: 4/5. Combining visual-textual evolution with model feedback targets a clear problem, although the components largely build on existing generation and RL tools.
• Experimental Thoroughness: 3/5. Two model sizes and multiple analyses are covered, but cost, independent label verification, and training schedules need clarification.
• Writing Quality: 3/5. The dual-loop concept is clear, but threshold and training-schedule descriptions require careful reading alongside the ablations.
• Value: 4/5. Useful for visual mathematical data construction and capability-driven curricula; extension to other visual tasks still requires validation.