Generating a Paracosm for Training-Free Zero-Shot Composed Image Retrieval¶

Conference: ECCV 2026
arXiv: 2602.00813
Code: https://github.com/leowangtong/Paracosm/
Area: Multimodal VLM
Keywords: Composed Image Retrieval, Zero-Shot Retrieval, Training-Free, LMM, Synthetic Images

TL;DR¶

Paracosm introduces the image editing capabilities of LMMs to zero-shot composed image retrieval for the first time. It directly generates a "mental image" for the multimodal query while simultaneously creating synthetic counterparts for the database images. By aligning the query and database into a unified synthetic space (paracosm) for matching, it substantially outperforms all previous zero-shot methods in a training-free manner, even rivaling supervised approaches.

Background & Motivation¶

Composed Image Retrieval (CIR) has emerged as an important direction in personalized visual search in recent years. Users provide a reference image and a modification text (e.g., "change the color of this dress from red to blue"), and the system is required to retrieve target images from a database that match this multimodal query. The core challenge of CIR is that the user's "mental image" is only implicit in the query and does not physically exist, yet the retrieval must be performed based on this non-existent image. Early methods relied heavily on manually annotated triplets (reference image, modification text, target image) for supervised training, which is costly and difficult to scale. Recently, zero-shot CIR (ZS-CIR) has become the mainstream paradigm. Among these, training-based methods map the reference image to pseudo-word tokens using text inversion networks and fuse them with the modification text for cross-modal matching. In contrast, training-free methods leverage Large Multimodal Models (LMMs) to generate a textual description for the multimodal query, thereby converting CIR into a text-to-image retrieval problem.

However, existing training-free methods possess a fundamental limitation: they substitute visual queries with textual descriptions. Regardless of how powerful the text generation capabilities of LMMs are, a textual description inevitably discards rich, fine-grained visual information, such as color gradients, material textures, spatial layouts, and lighting conditions—all of which are often critical for distinguishing closely resembling images during retrieval. Prior research has also attempted to use text-to-image (T2I) models to generate pseudo-target images for queries to assist matching. However, these methods still rely on an intermediate step of text description and fail to address the synthetic-to-real domain gap between generated images and real database images. When matching features extracted from synthetic images with real images, this domain discrepancy severely hinders similarity calculation.

The Key Insight of this work is: since LMMs already possess powerful image editing capabilities, why not directly "paint" the image that the user has in mind? Indeed, the reference image combined with the modification text perfectly matches the standard input format for image editing. Directly editing the reference image using an LMM is much more direct than generating an intermediate text description and then rendering it using a T2I model. Nonetheless, the edited "mental image" is still synthetic and does not belong to the same "domain" as the authentic photos in the database. To address this, rather than attempting to minimize the domain gap, this paper proposes to bring the database images into the synthetic space as well—by generating a synthetic counterpart for each database image, so that both the query and the database are compared within the same synthetic domain. Core Idea: Leverage LMMs to directly edit the reference image to generate the "mental image" of the multimodal query, while simultaneously synthesizing corresponding counterparts for each database image. By unifying the query and the database in a virtual synthetic space (paracosm) for similarity matching, training-free zero-shot composed image retrieval is achieved while significantly mitigating the synthetic-to-real domain gap.

Method¶

Overall Architecture¶

The core mechanism of Paracosm is remarkably straightforward: since a multimodal query implies a "mental image," this image is directly generated. Because the generated image is synthetic, the entire database is also reconstructed into synthetic equivalents so that matching can occur within the same synthetic domain. The entire pipeline is divided into two branches—query processing and database preprocessing—which ultimately converge in the feature space.

For each multimodal query (reference image + modification text), Paracosm utilizes the image editing capabilities of an LMM to directly edit the reference image based on the semantics of the modification text, yielding a "mental image." Then, another LMM generates a minimalist visual description for this mental image (focusing solely on content and suppressing stylistic or aesthetic details). The visual feature of the mental image, the textual feature of its description, and the textual feature of the original modification text are then weighted and fused to form the final query feature.

For the database, Paracosm pre-processes each image via a two-step procedure: first, an LMM generates a detailed description (covering all visible objects, attributes, and spatial relationships); second, a text-to-image (T2I) model generates a "synthetic counterpart" based on this description. During matching, the database representation is a simple sum of the visual features of the real image and its synthetic counterpart. Finally, cosine similarity is computed between the query feature and all database image features for retrieval.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    Q["Multimodal Query<br/>Reference Image + Modification Text"] --> A["LMM Image Editing<br/>Generate Mental Image"]
    A --> A2["LMM Generates<br/>Brief Visual Description"]
    Q --> B["Modification Text<br/>(Keep original text)"]
    A2 --> C["VLM Encoder<br/>Feature Extraction"]

    D["Database Image"] --> E["LMM Generates<br/>Detailed Description"]
    E --> F["T2I Model<br/>Generate Synthetic Counterpart"]
    D --> G["VLM Visual Encoding"]
    F --> G

    C --> H["Weighted Fusion of Query Features"]
    G --> I["Database Image Features<br/>Sum of Real + Synthetic Features"]
    H --> J["Cosine Similarity<br/>Ranking & Retrieval"]
    I --> J
    J --> K["Return Top-K<br/>Target Images"]

Key Designs¶

1. Mental Image Generation: Substituting Textual Descriptions with LMM Image Editing

Existing training-free ZS-CIR methods (e.g., CIReVL, LDRE, OSrCIR) rely on LMMs to generate text descriptions and subsequently match these descriptions with database images—essentially reducing the problem to text-to-image retrieval, which discards substantial visual information. Paracosm addresses this limitation by recognizing that the input of a multimodal query (reference image + modification text) perfectly matches the input format of an image editing task. Rather than using the roundabout path of "writing a description and retrieving based on it," it is far more direct to leverage the editing capabilities of an LMM to directly paint the "mental image." Specifically, Paracosm feeds the reference image and the modification text into Qwen-Image-Edit (an LMM with image editing support) to directly output the edited mental image. Empirical results show that this direct editing approach significantly outperforms the indirect pipeline of "writing a description first and then generating a pseudo-target image with a T2I model" (Table 1: direct editing achieves 32.27% on CIRR R@1 vs. 31.71% for T2I generation). The mental image preserves abundant visual details that text descriptions fail to convey—such as subtle shifts in material, lighting, and shape—which are critical fine-grained cues for distinguishing similar candidate images.

2. Database Synthetic Counterparts: Aligning the Matching Process in the Synthetic Space

Directly matching synthetic mental images with genuine database images suffers from a prominent domain gap. Because the feature distributions of the two domains differ, cosine similarity is easily corrupted by domain shifts rather than being dominated by purely semantic differences. Paracosm addresses this via a clever strategy: instead of attempting to narrow the domain gap (which typically requires domain adaptation training), it migrates the database to the synthetic domain as well. For each database image, it performs two operations: it first utilizes an LMM to generate an exhaustive description (capturing all visible objects, attributes, spatial properties, and fine-grained visual elements), and subsequently uses this description as a prompt for Qwen-Image (or T2I models like FLUX or LongCat) to generate a synthetic counterpart. During matching, the database representation is a sum of the visual features of the real image and its synthetic counterpart (with equal weights, i.e., \(\phi^i = V(\mathbf{I}^i) + V(\mathbf{I}^i_{syn})\)). Ablation studies demonstrate that introducing the synthetic counterparts boosts CIRR R@1 from 27.93% to 32.27% and CIRCO mAP@5 from 18.29 to 26.10, indicating that the performance gain achieved by aligning the matching workspace within a single domain far outweighs any noise introduced by the synthetic images.

3. Three-Way Weighted Fusion of Multimodal Query Features

Instead of relying solely on the mental image at the query end, Paracosm elegantly fuses three distinct information sources: the visual encoding of the mental image \(V(\mathbf{I}_{mental})\), the text encoding \(T(\mathbf{t}_{query})\) of the mental image's brief description \(\mathbf{t}_{query}\), and the text encoding \(T(\mathbf{t}_{mod})\) of the original modification text \(\mathbf{t}_{mod}\). These components are mathematically combined using a weighting hyperparameter \(\lambda\):

\[\small\mathbf{q} = \lambda\big(V(\mathbf{I}_{mental}) + T(\mathbf{t}_{query})\big) + (1-\lambda)T(\mathbf{t}_{mod})\]

The mental image and its brief description are weighted by \(\lambda\) and summed to establish a hybrid "visual-textual base," while the modification text is incorporated independently with a weight of \(1-\lambda\). The intuition is that the mental image carries rich visual details but may lack precision in specific semantic attributes; the brief description provides explicit semantic anchoring; and the modification text preserves the explicit modification intent specified by the user in the original query. These three elements complement each other. Experiments reveal that \(\lambda=0.3\) is consistently optimal across all three benchmark datasets. This implies that the weight of the modification text (0.7) is unexpectedly higher than the combined weight of the mental image and its description (0.3), highlighting that the stability and reliability of textual information remain vital for retrieval in the presence of domain gaps.

Loss & Training¶

Paracosm is an entirely training-free method, requiring no loss functions or training procedures. All modules (LMM-based image editing, LMM-based description generation, T2I synthetic image generation, and VLM feature extraction) utilize off-the-shelf pretrained models. The only "tuning" involves a grid search for the hyperparameters \(\lambda\) (modification text weight) and \(\beta\) (real vs. synthetic image feature weight) on the CIRR validation set, fixing them at 0.3 and 0.5, respectively.

Key Experimental Results¶

Main Results¶

Paracosm is evaluated on three standard benchmarks (CIRR, CIRCO, and Fashion IQ) using two VLM backbones, CLIP ViT-L/14 and OpenCLIP ViT-G/14, and compared comprehensively against state-of-the-art zero-shot methods.

CIRR and CIRCO Test Set Results (ViT-L/14 backbone):

Method	CIRR R@1	CIRR R@5	CIRR RSubset@1	CIRCO mAP@5	CIRCO mAP@50
Pic2Word (CVPR'23)	23.90	51.70	53.76	8.72	11.29
SEARLE (ICCV'23)	24.24	52.48	53.76	11.68	15.12
LinCIR (CVPR'24)	25.04	53.25	57.11	12.59	15.85
LDRE (SIGIR'24)	26.53	55.57	60.43	23.35	27.50
CIReVL (ICLR'24)	24.55	52.31	59.54	18.57	21.80
IP-CIR+LDRE (CVPR'25)	29.76	58.82	62.48	26.43	31.07
CIG+SEARLE (CVPR'25)	26.72	55.52	57.95	12.84	16.17
Paracosm (ours)	31.95	61.56	64.68	30.24	35.42

Fashion IQ Validation Set Results (ViT-L/14 backbone):

Method	Shirt R@10	Dress R@10	Toptee R@10	Average R@10
Pic2Word	26.20	20.00	27.90	24.70
SEARLE	26.89	20.48	29.32	25.56
LinCIR	29.10	20.92	28.81	26.28
CIReVL	29.49	24.79	31.36	28.55
CIG+LinCIR	28.90	21.12	29.78	26.60
Paracosm (ours)	31.80	24.99	31.82	29.45

Paracosm significantly outperforms all existing zero-shot methods across all evaluation metrics. When equipped with a stronger ViT-G/14 backbone, its superiority is further amplified—achieving 39.82 on CIRCO mAP@5 (compared to only 30.47 of the previous best method, OSrCIR) and 39.30 on CIRR R@1 (compared to 37.26 of OSrCIR). Even when compared with supervised methods, Paracosm matches or exceeds their performance across multiple metrics.

Ablation Study¶

Complete ablation studies on CIRR and CIRCO based on the CLIP ViT-B/32 backbone (Table 6):

Configuration	Query Encoding	Database Encoding	CIRR R@1	CIRCO mAP@5
Baseline: Mental image description only	\(\mathbf{t}_{query}\)	Real images	17.21	14.91
+ Mental image	\(\mathbf{t}_{query}+\mathbf{I}_{mental}\)	Real images	18.80	13.71
+ Mental image + Modification text	\(\mathbf{t}_{query}+\mathbf{I}_{mental}+\mathbf{t}_{mod}\)	Real images	27.93	18.29
+ Synthetic counterparts (Full model)	Full query features	Real + Synthetic	32.27	26.10

Key Findings¶

Mental image contribution: Significant but not decisive: Simply adding the mental image (while omitting the modification text) actually leads to a slight decline on CIRCO. This indicates that the mental image must be combined with the modification text to yield retrieval gains; while the visual details are abundant, the semantics may lack precision, which the text resolves by providing a clear modification direction.
Synthetic counterparts: The greatest source of performance gain: Incorporating synthetic counterparts boosts CIRCO mAP@5 from 18.29 to 26.10 (+42.7%), vastly exceeding the contribution of any other individual component. This validates the core assumption of a "unified matching space"—that the benefits of mitigating the domain gap far outweigh any noise introduced by synthesis.
Modification text weight \(\lambda=0.3\) is consistently optimal across datasets: Although CIRR, CIRCO, and Fashion IQ represent non-overlapping benchmarks, the optimal \(\lambda\) consistently remains at 0.3. This indicates that the modification text should play a dominant role (0.7) in query features relative to the visual base (0.3), suggesting that the quality of images generated by current LMMs still leaves room for improvement.
Equal weighting (\(\beta=0.5\)) is simple yet effective: Allocating equal weight to both empirical real images and synthetic counterparts yields optimal performance, while avoiding the parameter-tuning complexities of asymmetric weighting.
Robust to choice of LMM: Switching between Qwen-Image-Edit and LongCat-Image-Edit yields stable performance (32.27 vs 32.12 on CIRR R@1), demonstrating that the efficacy of Paracosm stems from its holistic framework rather than individual models.
Negligible gains compared to GPT-4o: Replacing Qwen2.5-VL with GPT-4o to generate mental image descriptions leads to a negligible shift in performance (from 32.27 to 31.86, a slight drop), indicating that the core method does not rely heavily on the scale of the LMM.

Highlights & Insights¶

An "if you can't beat them, join them" approach to domain adaptation: Rather than seeking to minimize the synthetic-to-real domain gap (which typically involves domain shift minimization or alignment), Paracosm takes the opposite route—pulling the database into the synthetic space to ensure matching occurs within the same domain. This elegant, training-free approach is highly relevant for other tasks involving synthetic-to-real domain matching (e.g., neural rendering retrieval, sim-to-real domain cross-retrieval).
Image editing naturally fits CIR queries: The multimodal query format of CIR (reference image + editing instruction) aligns perfectly with the inputs of image editing tasks. Paracosm is the first work to systematically exploit this alignment—not as a coincidence, but as an insight into the intrinsic structural nature of the task.
Design wisdom of three-way feature fusion: Combining the mental image (rich but imprecise visuals), brief description (semantic anchor), and modification text (original constraint) forms a highly complementary system. The finding that \(\lambda=0.3\) is optimal reveals a counter-intuitive fact: in CIR, the reliability of textual information can be more critical than the richness of images.
Training-free yet superior to supervised models: Without being trained on any CIR datasets, Paracosm surpasses supervised methodologies such as Combiner and BLIP4CIR on multiple metrics (e.g., CIRCO mAP@50), demonstrating the profound potential of zero-shot generalization in foundation models.

Limitations & Future Work¶

Highly sensitive to LMM generation quality: When the mental image edited by the LMM deviates from reality (e.g., rendering a cartoon-style duck instead of a realistic plush toy), retrieval failure is inevitable. Figure 7 illustrates several failure cases, including counterfactual generation (burners on an oven door), failed editing (failing to alter freezer door color), and query misunderstanding. Future work could introduce consistency-checking mechanisms to filter or rectify erroneous generations.
High computational overhead: Processing 123K database images in CIRCO takes approximately 12.9 hours using a cluster of 16 A100 GPUs, requiring ~41.4 GB of physical storage. Although the synthetic features are eventually compressed to 0.38 GB and preprocessed offline only once, active deployments still must balance retrieval performance against computational cost.
Manual tuning of prompt templates across datasets: Due to slight variations in modification text formatting between CIRR, CIRCO, and Fashion IQ (e.g., CIRCO contains a "shared concept"), Paracosm design requires slightly tailored prompt templates for each benchmark. Future iterations could explore adaptive prompt generation, enabling LMMs to automatically construct optimal prompts based on input templates.
Lack of filtering mechanisms for inappropriate queries: As noted in the impact analysis, Paracosm lacks mechanisms to detect or reject malicious queries. If users input inappropriate modification text, intermediate mental images and final retrieval results could potentially lead to negative social impacts.

vs. OSrCIR (Tang et al., CVPR'25): While both are training-free ZS-CIR methods, OSrCIR uses a reflective chain-of-thought to raise the quality of LMM-generated descriptions, which fundamentally remains a text \(\rightarrow\) image retrieval methodology. Paracosm, conversely, directly generates a mental image and employs synthetic counterparts to resolve the domain gap—a fundamental paradigm shift. Under the ViT-G/14 backbone, Paracosm reaches 39.82 on CIRCO mAP@5, whereas OSrCIR only achieves 30.47. Furthermore, rigorous reproduction in this work revealed that OSrCIR likely utilized OpenCLIP rather than the claimed CLIP, highlighting reproducibility issues in the community.
vs. IP-CIR (Li et al., CVPR'25): Although IP-CIR also attempts to generate pseudo-target images to aid retrieval, it follows an indirect path of "description \(\rightarrow\) LLM modified description \(\rightarrow\) T2I generation," and must be combined with text inversion networks like LDRE. Paracosm directly edits the reference image and shifts the database into the synthetic space, offering a simpler pipeline without additional training.
vs. LDRE (Yang et al., SIGIR'24): LDRE boosts recall by generating multiple diverse descriptions, yet all descriptions are purely textual and discard fine-grained visual details. Paracosm outperforms LDRE's multi-description strategy using only a single mental image (CIRR R@1: 39.30 vs. 36.15).

Rating¶

Novelty: ⭐⭐⭐⭐⭐ First to systematically utilize LMM image editing capabilities and synthetic counterpart strategies to address zero-shot CIR. The strategy is clean and elegant, presenting a paradigm shift in CIR methodology.
Experimental Thoroughness: ⭐⭐⭐⭐⭐ Extensively validated against all major zero-shot baselines across 3 datasets and 2 backbones. Ablation studies comprehensively cover all components, augmented by analysis of computational costs, LMM robustness, cross-generator comparisons, and critical discussion on existing SOTA reproducibility issues.
Writing Quality: ⭐⭐⭐⭐⭐ Motivations are clearly derived from first principles with tight logical progression, rich illustrations, and exhaustive empirical analysis (e.g., the discovery of the universally optimal \(\lambda\)). Limitations are also candidly reflected upon.
Value: ⭐⭐⭐⭐⭐ Substantially elevates the zero-shot CIR SOTA. The method is training-free, easily deployable, and carries immense practical value for real-world scenarios like e-commerce and fashion search.