Correlation-Weighted Multi-Reward Optimization for Compositional Generation¶

Conference: ECCV 2026
Paper: ECCV
Area: Image Generation
Keywords: compositional generation / multi-reward RL / GRPO / correlation-based reweighting / text-to-image

TL;DR¶

To fix a key bottleneck of multi-concept compositional generation — naive aggregation of concept-wise rewards lets easy concepts dominate optimization while genuinely conflicting concepts get diluted — CMO computes a Pearson correlation matrix over concept rewards inside a sample group, estimates each concept's difficulty from its average correlation with the others, and reweights conflicting concepts by \(\text{softmax}(1-\alpha)\); plugged into a MixGRPO + GDPO RL recipe for fine-tuning SD3.5 and FLUX.1-dev, it delivers consistent gains on ConceptMix, GenEval 2 and T2I-CompBench.

Background & Motivation¶

Text-to-image diffusion and flow-matching models (SD3.5, FLUX.1-dev and friends) are already reliable at single-concept generation, yet compositional generation remains their weak spot: a real prompt typically demands several concepts at once, with every attribute bound to the right object and with counts and spatial relations satisfied. Early work mitigated token entanglement at inference time through attention control or extra conditioning signals — structured diffusion guidance, Attend-and-Excite and token merging all belong to this family — and DiT / flow-matching backbones improved token separation enough to partially relieve the problem, but once a prompt packs five or six concepts the model still drops some of them. The more recent mainstream route treats an evaluation metric such as GenEval as a reward and post-trains with diffusion RL (FlowGRPO, MixGRPO, Pref-GRPO, IterComp), which works well in single- or two-concept settings.

The difficulty lies in how the rewards are aggregated in the multi-concept regime. Facing a complex prompt, the model's typical failure is not total failure but partial success: within one sampling group, some images get the objects right but the count wrong, others get the color right while losing the subject. The concept rewards then pull against each other, and existing methods simply aggregate them naively — GRPO sums them directly, while GDPO uses decoupled normalization to fix the interference between multiple rewards' normalization (reward collapse) but still adds them with equal weights afterwards. Equal weighting lets optimization be dominated by concepts that are almost always satisfied within a group, while the hard concepts that most need gradient are averaged away. Worse, some concepts are negatively correlated: getting the gray color right often comes at the cost of losing the spider or the hammers — this is a genuine generation conflict. The paper first nails this down empirically: on ConceptMix it computes correlations among concept rewards over 10 images generated from a single prompt, and plots each method's ratio of negatively correlated concept pairs against its Full Mark score. The three points for SD3.5, SD3.5 + FlowGRPO and SD3.5 + CMO land around 0.267 / 0.313 / 0.391 (consistent with the SD3.5-M baseline and the full model in Table 5), so more negative correlation means a lower score; moreover, as the number of concepts K grows, the baselines' ratio of negatively correlated concept pairs climbs sharply, showing that naive aggregation amplifies the trade-offs between concepts. The remaining obstacle is that which concepts are hard varies with the prompt combination — "three hammers" is a hard concept in some prompts and not in others — which makes manual weight tuning impractical.

Core idea: since "hard" shows up as the statistical correlation among concept rewards inside a sample group, turn manual weight tuning into automatic difficulty estimation on the correlation matrix — the more negatively correlated (i.e. the more conflicting) a concept is with the others, the larger its weight — and apply \(\text{softmax}(1-\text{average correlation})\) to the GDPO-normalized advantages so that the model satisfies all concepts in the prompt simultaneously.

Method¶

Overall Architecture¶

CMO is a multi-reward reinforcement learning post-training framework for multi-concept prompts. Its input is one multi-concept prompt plus a batch of images the policy model generates for it, and its output is an updated LoRA policy. The pipeline has four steps: first the prompt is parsed into a structured "multi-concept configuration", decomposing the objects, colors, textures, shapes, styles, sizes, counts and spatial relations it mentions into K concept groups; then the T2I model (SD3.5 or FLUX.1-dev) samples G images for that prompt and a set of dedicated reward functions scores each of these concepts, stacking the results into a \(G \times K\) multi-reward matrix; next, pairwise Pearson correlations are computed across concepts on that matrix to obtain a difficulty score \(\alpha_k\) per concept, which a softmax turns into a weight \(w_k\); finally the weights multiply the GDPO decoupled-normalized group-relative advantages, the weighted sum is batch-normalized into the final advantage, and the policy is updated with it. Optimization uses MixGRPO-style mixed ODE–SDE sampling, restricting stochastic SDE exploration to the first few timesteps while the remaining steps take deterministic ODE updates, which removes the computational overhead of full-step SDE sampling.

The central object of the method is that \(G \times K\) multi-reward matrix: every concept-level reward is computed and normalized independently, and correlation-based reweighting happens at the level of the already-normalized advantages, so CMO replaces GDPO's equal-weight summation rather than introducing a separate optimization objective.

Key Designs¶

1. Multi-concept reward decomposition: split a prompt into 8 concept groups, each with its own reward function

Weighting concepts presupposes an independent reward signal per concept, so CMO starts by designing a fine-grained reward decomposition instead of a single scalar scorer. Object existence uses SAM 3 for text-guided instance detection: the object name is fed in as the detection prompt, and the resulting bounding boxes and masks double as the shared localization layer for every downstream localized evaluation — crucially, this isolates the target entity from background noise, so the attribute, count, size and spatial rewards all share one set of localizations instead of each running its own detector with its own noise. The attribute reward (color, texture, shape, style) is measured with the zero-shot classifier OpenCLIP-H: on the region cropped by the target object's bounding box, the image feature is compared against the normalized text feature of every candidate attribute word in that category's vocabulary (e.g. red, green, blue), and a softmax over those similarities yields the probability of the attribute \(v^*\) named in the prompt:

\[r_i^{\text{attr}}=\frac{\exp(\langle f_{\text{img}}, f_{v^*}\rangle)}{\sum_{v \in \mathcal{V}_k}\exp(\langle f_{\text{img}}, f_v\rangle)}\]

Keeping this reward continuous rather than a 0/1 verdict is what lets it supply distinguishable gradients early in training. The numeracy reward deliberately avoids a binary existence-style penalty and instead uses a differentiable heuristic built on count deviation: the further the detected count \(c_i\) is from the target count \(c_i^*\), the smaller the reward (⚠️ the cached text of this equation is corrupted; refer to the original paper for its exact form). Size attributes are scored by an inverse-squared decay over the rank of every detected object's area. Spatial relations come in two branches: 2D relations (above, below, left, right) use geometric heuristics that award graded partial scores of 1.0 / 0.5 / 0.0, while 3D depth relations (in front of, behind) use Depth Anything 3 to estimate a depth map \(\mathcal{D}(x)\) and take the mean depth \(\bar{d}_i\) over the pixels inside the target mask, requiring \(\bar{d}_i\) to be clearly smaller than \(\bar{d}_j\) for "in front of" (with a tolerance \(\epsilon\) absorbing depth-estimation noise) and the opposite condition for "behind". The point of the whole design is that each concept is independently scorable and returns a continuous partial score, so that each column of the reward matrix really does encode how well that concept is satisfied in that image — which is exactly what the later correlation analysis needs.

2. Correlation-based difficulty estimation and reweighting: locating mutually conflicting concepts with the in-group reward correlation matrix

This is the paper's core contribution. For the G images sampled from one prompt, the K concepts' rewards form a matrix \(R^{(i)} \in \mathbb{R}^{G \times K}\), and the pairwise Pearson correlation matrix \(C \in \mathbb{R}^{K \times K}\) is computed as

\[C_{k,l}=\text{Corr}\big(R^{(i)}_{k}, R^{(i)}_{l}\big)\]

Each concept then receives a difficulty score \(\alpha_k\) defined as the average of its correlations with all other concepts, \(\alpha_k=\frac{1}{K-1}\sum_{l\neq k}C_{k,l}\). Since \(C_{k,l}\in[-1,1]\), a concept that is negatively correlated with the others gets a low \(\alpha_k\), meaning "getting it right tends to go together with getting another concept wrong" — i.e. it is the hardest to satisfy jointly, whereas highly correlated concepts are the easy ones that rise and fall together. The weight comes from \(\text{softmax}(1-\alpha_k)\), where \(1-\alpha_k\) flips "harder means larger weight" into the right direction, and it replaces GDPO's equal-weight summation:

\[\hat{A}^{(i,j)}_{\text{total}}=\frac{\sum_k w_k A^{(i,j)}_k-\mu_B}{\sigma_B+\epsilon},\qquad A^{(i,j)}_k=\frac{r^{(i,j)}_k-\mu^{(i)}_k}{\sigma^{(i)}_k}\]

Here \(A_k\) is the group-relative advantage of the k-th reward and \(\mu_B,\sigma_B\) are the batch-level statistics of the weighted sum (⚠️ Eq. 11 is corrupted in the cached text; this form is restored from Eq. 2 and Figure 2 as "weighted sum, then batch normalization" — refer to the original paper). Two bypasses keep training stable: if a concept has near-zero variance within the group (satisfied in every image, or in none), its correlation is mathematically undefined, so \(\alpha_k\) is set to the default maximum correlation of 1.0 to avoid an unstable gradient from division by zero; and if the sampling group contains incomplete padding values, correlation estimation is skipped for the whole group. The first bypass dovetails naturally with GDPO's group normalization — a zero-variance concept already has a vanishing group-relative advantage, so down-weighting it costs no useful gradient while concentrating the gradient budget on genuinely conflicting concepts. Compared with equal-weight multi-reward schemes and manual weight tuning, this mechanism differs in three ways: difficulty is instantiated per group, so the same concept carries different weights under different prompt combinations; difficulty is determined by correlation rather than by reward magnitude, since a low reward may merely mean the policy has not learned the concept yet, whereas only negative correlation indicates a real optimization conflict; and no new parameters or difficulty predictor are introduced.

3. Difficulty-oriented training-data construction: mixing single-attribute with multi-concept prompts and skewing the concept-group sampling ratio

The structural density of the multi-concept rewards needs a matching training distribution to supervise, otherwise the best reward has nothing to act on. CMO's training set holds 5k prompts, half of them 2.5k complex multi-concept prompts (synthesized with the ConceptMix pipeline plus Qwen3-30B, binding up to eight concepts per entity, i.e. \(K+1=8\)) and the other half 2.5k single-attribute prompts ({color} {object}-style clean bindings); the latter follows the observation of FlowGRPO and TempFlow-GRPO that a single-attribute set improves overall compositional ability. More importantly, the sampling ratio over concept groups is deliberately skewed to Color : Texture : Shape : Size : Spatial : Numeracy = 3 : 3 : 2 : 1 : 10 : 7, exposing the two hardest categories — spatial relations and numeracy — over and over during training. The ablation confirms that this data construction is an independent contribution: base optimization alone (DRO) barely moves the needle, but swapping in this data lifts Avg Full Mark from 0.2713 to 0.3305 (single-attribute only) and then to 0.3531 (with multi-concept), the largest gain outside correlation reweighting. The authors acknowledge that the ratio is fixed a priori and propose in the discussion that it should instead be adapted from observed failure patterns.

A Worked Example¶

Take the prompt in Figure 1: "a single, gray, circle-shaped spider, and exactly three hammers, clearly visible". Evaluation breaks it into five questions: Q1 are there hammers, Q2 is the spider circle-shaped, Q3 is there a spider, Q4 is the spider gray, Q5 does the image contain three hammers. A baseline model's typical pattern of partial success over one sampling group is: Q1 almost always correct (scoring 1 + 1 = 2), Q2 always wrong (0 + 0 = 0), Q3 correct in only a few images (0.2), Q4 partially correct (0.5), Q5 correct in most images (0.8). After in-group normalization these concept rewards become advantages of roughly 1.8 / −0.9 / −0.7 / −0.2 / 0.1, while correlation reweighting assigns weights of 0.1, 0.1, 0.8, 0.9 and 0.8 to Q1–Q5, so the final advantages are weight × normalized advantage = 0.18 / −0.09 / −0.56 / −0.18 / 0.08 (Figure 1 gives illustrative values). The point: Q1, a concept handed over for free, is pressed down to 0.1, whereas Q4 ("getting the gray right may cost you the spider", 0.9) and Q5 ("must produce three hammers", 0.8) are pushed up the most, and Q3's large negative gradient becomes the group's main optimization target because its weight is amplified. The weights do not follow reward magnitude; they follow whether a concept conflicts with the others — that is what correlation reweighting does on one concrete sampling group.

Loss & Training¶

The base optimization recipe, which the authors call DRO (Decoupled Reward Optimization), is MixGRPO plus GDPO. On the sampling side, to avoid the cost of full-step SDE, stochastic exploration is confined to a sliding window \(S \subseteq [0,T]\) while the steps outside it take deterministic ODE updates:

\[ dx_t=\begin{cases}\big[v_t-\tfrac{1}{2}g^2(t)s_t\big]dt+g(t)dw, & t\in S\\ v_t\,dt, & t\notin S\end{cases} \]

where \(v_t\) and \(s_t\) are the velocity field and score function; gradients are thus concentrated inside \(S\) and the number of optimization timesteps shrinks considerably. Following MixGRPO, the SDE updates here are restricted strictly to the first 3 timesteps. On the multi-reward side, GDPO's decoupled normalization first computes each of the K rewards' group-relative advantage independently, \(A^{(i,j)}_k=(r^{(i,j)}_k-\mu^{(i)}_k)/\sigma^{(i)}_k\), so that pre-normalization aggregation cannot collapse distinct reward combinations onto identical advantages; CMO then replaces the equal-weight sum that follows with a \(w_k\)-weighted sum and batch-normalizes it into \(\hat{A}_{\text{total}}\). Training details: LoRA fine-tuning of SD3.5 and FLUX.1-dev, learning rate \(3\times10^{-4}\), each iteration sampling 8 prompts and 16 images per prompt on 4 GPUs for a global batch size of 512, 10 sampling steps for SD3.5 and 6 for FLUX.1-dev, together with Coefficient Preserving Sampling (CPS) to balance exploration and efficiency.

Key Experimental Results¶

Main Results¶

Three compositional-generation benchmarks: ConceptMix (strict multi-concept; Full Mark and Concept Fraction are reported, where k is the number of concepts and each prompt contains about k+1 concepts), GenEval 2 (Soft-TIFA at atom level, TIFA_AM, and prompt level, TIFA_GM), and T2I-CompBench (attribute binding, spatial / non-spatial relations, numeracy, complex composition, and their average). On all benchmarks 10 images per prompt are generated with different random seeds and the metrics are averaged. ConceptMix's evaluator is switched from the original paper's GPT-4o to Qwen3-VL-8B, and all baselines are re-evaluated under exactly this setup, so the absolute numbers are not directly comparable with published figures.

Model (metric)	k=1	k=2	k=3	k=4	k=5	k=6	k=7
FLUX.1-dev (Full Mark)	0.6647	0.4127	0.2680	0.1620	0.1173	0.0567	0.0372
FlowGRPO (Full Mark)	0.7600	0.5457	0.3517	0.2103	0.1727	0.0827	0.0713
Pref-GRPO (Full Mark)	0.7037	0.4790	0.3023	0.2120	0.1603	0.0897	0.0647
Qwen-Image (Full Mark)	0.8183	0.6583	0.5060	0.3803	0.3353	0.2213	0.1747
CMO (FLUX.1-dev, Full Mark)	0.8410	0.7063	0.5700	0.3947	0.3560	0.2317	0.1883
FLUX.1-dev (Concept Frac.)	0.8162	0.7430	0.7093	0.6706	0.6779	0.6467	0.6576
FlowGRPO (Concept Frac.)	0.8747	0.8144	0.7726	0.7235	0.7263	0.7010	0.6953
Qwen-Image (Concept Frac.)	0.9052	0.8600	0.8403	0.8193	0.8171	0.7869	0.7942
CMO (FLUX.1-dev, Concept Frac.)	0.9147	0.8850	0.8635	0.8188	0.8214	0.7917	0.7884

Model	Color	Shape	Texture	Spatial	Non-Spatial	Numeracy	Complex	Avg
FLUX.1-dev	0.7358	0.4802	0.5989	0.2461	0.3067	0.6107	0.4281	0.4866
FlowGRPO	0.8263	0.6177	0.7241	0.5237	0.3184	0.6987	0.3905	0.5856
Qwen-Image	0.8395	0.5882	0.7407	0.4454	0.3136	0.7553	0.4414	0.5892
SD3.5 + CMO	0.8692	0.6427	0.7959	0.5475	0.3201	0.7200	0.4036	0.6141
FLUX.1-dev + CMO	0.8607	0.6188	0.7206	0.4577	0.3171	0.7066	0.4909	0.5961

GenEval 2 (Soft-TIFA)	TIFA_AM ↑	TIFA_GM ↑
FLUX.1-dev	64.1	17.1
FlowGRPO	70.9	21.8
Pref-GRPO	70.3	23.0
Qwen-Image	80.0	31.4
CMO	80.0	34.0

Ablation Study¶

DRO denotes the base multi-reward optimization (MixGRPO + GDPO); Single-Attr. and Multi-Concept denote whether the corresponding training set is included; CR is correlation-based reweighting.

Config	DRO	Single-Attr.	Multi-Concept	CR	Avg. Full Mark	Avg. Concept Frac.
SD3.5-M (Base)	–	–	–	–	0.2667	0.7203
+ DRO	✓	×	×	×	0.2713	0.7411
+ DRO + single-attribute data	✓	✓	×	×	0.3305	0.7752
+ DRO + single-attribute + multi-concept	✓	✓	✓	×	0.3531	0.7993
Full CMO	✓	✓	✓	✓	0.3913	0.8126

Key Findings¶

The harder the task, the larger the gain. At k=7 CMO's Full Mark reaches 0.1883, roughly 5× the FLUX.1-dev base (0.0372) and above Qwen-Image (0.1747); Concept Fraction rises from the base's 0.6576 to 0.7884, indicating the improvements come mainly from preventing whole objects from being dropped rather than from gaming any single attribute.
Correlation reweighting is the largest single contributor. With data and base optimization already in place (0.3531), adding CR alone pushes the score to 0.3913 and Concept Fraction from 0.7993 to 0.8126; conversely, adding DRO alone barely helps (0.2667 → 0.2713), so multi-reward plus decoupled normalization is not enough on its own to resolve concept conflicts.
Quantitative evidence that conflicts grow with complexity. The baselines' ratio of negatively correlated concept pairs rises rapidly with the number of concepts K, whereas CMO keeps it low; the left panel of Figure 3 places that ratio next to the Full Mark score and shows a clear inverse relation, directly supporting the motivating claim that optimization conflicts cause dropped concepts.
Ties and losses, reported honestly. On GenEval 2's atom-level TIFA_AM, CMO ties Qwen-Image (80.0 vs 80.0), with its edge confined to prompt-level TIFA_GM (34.0 vs 31.4); on T2I-CompBench's Numeracy, Qwen-Image's 0.7553 still beats CMO's two results (0.7200 / 0.7066); Non-Spatial uses CLIPScore, where all methods crowd into 0.3067–0.3201, so that dimension is close to saturation and does not constitute real evidence.
Both backbones benefit. SD3.5 + CMO attains the best T2I-CompBench Avg (0.6141) with leads in attribute binding and spatial relations, while FLUX.1-dev + CMO is strongest on Complex (0.4909), showing the method is not tied to one backbone.

Highlights & Insights¶

Defining "hard" on the correlation structure rather than on reward magnitude. A low reward may simply mean the policy has not learned the concept yet; only negative correlation reveals that two concepts repel each other during optimization. This distinction keeps the difficulty signal free of the noise from "this concept is intrinsically difficult", and turns "which concepts conflict" into a quantity computable with no extra training (a Pearson correlation on one \(G\times K\) matrix). The view transfers directly to any multi-objective alignment setting, e.g. multi-goal agent RL or multidimensional preference weighting in RLHF.
The zero-variance bypass is a self-consistent detail. For a concept satisfied in every image of the group or in none, correlation is undefined; setting it to the default maximum correlation of 1.0 down-weights it, and because such a concept already has a vanishing group-relative advantage after normalization, down-weighting costs no useful gradient while avoiding a divide-by-zero gradient.
bbox / mask reuse as a shared localization layer. The boxes and masks from SAM 3 simultaneously serve the attribute (cropped region for OpenCLIP comparison), count (number of boxes), size (ranked box areas) and spatial (2D geometry plus 3D depth order) rewards, so a single localization error does not resurface in four different guises across four rewards — an engineering detail of multi-reward design that is easy to overlook.
The 3D relation reward compares depth order with a tolerance. Averaging depth inside two masks using Depth Anything 3's depth maps and allowing a tolerance \(\epsilon\) for monocular depth noise yields a spatial-relation supervision signal that needs no 3D annotation yet stays closer to the semantics of "in front of" than detector-box IoU heuristics.

Limitations & Future Work¶

The training-data sampling ratio (Color : Texture : Shape : Size : Spatial : Numeracy = 3 : 3 : 2 : 1 : 10 : 7) is a fixed, hand-set prior; the authors list this first among future directions, proposing that failure patterns should drive it adaptively and that the prompt distribution itself should enter the difficulty-aware optimization loop.
Correlations are estimated only within one prompt's sampling group, and Pearson coefficients are noisy when G is small; whether cross-group or cross-prompt-pool estimation gives a more stable difficulty signal is not tested.
The ablation runs only on one line, SD3.5-M (Table 5), and is not repeated on FLUX.1-dev, while the main tables report CMO on FLUX.1-dev — so the ablation conclusions and the main results do not use exactly the same backbone.
The training scale is modest: LoRA fine-tuning, 4 GPUs, 5k prompts, with no validation on full-parameter fine-tuning or autoregressive generative models, and no report of training time or memory cost — even though each step must run SAM 3, OpenCLIP, Depth Anything 3 and a VLM evaluator, so reward computation is far from free.
There is a comparability caveat on evaluation: ConceptMix's evaluator is changed from GPT-4o to Qwen3-VL-8B (internally fair since all baselines are re-run), and no gap is opened on GenEval 2's TIFA_AM.
The cached full text excludes the appendix, which holds the exact 2D spatial-relation formulations and the construction / normalization details of the attribute candidate vocabulary; the numeracy reward equation is corrupted in the cached text (⚠️ refer to the original paper), so a full check of these details requires the appendix.

vs FlowGRPO / MixGRPO: they bring group-relative advantages to RL alignment of diffusion / flow models, and in multi-reward settings they sum rewards with equal weights. This paper argues that equal weighting systematically under-rates conflicting concepts, and keeps the efficiency recipe intact (MixGRPO's mixed ODE–SDE) while replacing only the aggregation, so the two lines of improvement are orthogonal and composable.
vs GDPO: GDPO addresses interference between multiple rewards' normalization that causes reward collapse, taking the route of normalizing each reward separately before aggregating — but the aggregation is still equal-weight. CMO adopts it as the base (DRO in the ablation) and swaps only the final equal-weight sum for correlation weighting, so it can be read as a weighted-aggregation version of GDPO.
vs IterComp / Pref-GRPO: IterComp iteratively learns composition-aware feedback from a model gallery and Pref-GRPO stabilizes GRPO with pairwise preference rewards; both work on the surface form of the reward and do not explicitly model relations between concepts, so they remain constrained by the same conflict problem as concept count grows (at k=7 on ConceptMix they reach only 0.0040 and 0.0647 respectively).
vs training-free attention control (Attend-and-Excite, structured diffusion guidance, token merging): these modify attention or token interaction at inference time, need no training and drop onto any model, but they face the binding-ability ceiling already fixed by large-scale pretraining, improve only marginally, and cannot exploit global information such as which concepts conflict; this paper chooses post-training instead, at the cost of needing model weights and paying for reward computation.
Transferable insight: within multi-reward weighting, using the correlation between rewards rather than their mean or variance to define "hard" is an operation that costs almost nothing extra (one \(K\times K\) correlation) yet changes the gradient allocation. The same idea could be tried on multidimensional preferences in RLHF (helpfulness and harmlessness often conflict), on task-conflict weighting in multi-task learning, and on multi-constraint reward design for agents.

Rating¶

Novelty: ⭐⭐⭐⭐ The angle of estimating difficulty from the correlation structure of concept rewards and reweighting accordingly is new, but MixGRPO, GDPO, SAM 3 and OpenCLIP are all off-the-shelf components, making this combinatorial innovation.
Experimental Thoroughness: ⭐⭐⭐⭐ Three compositional benchmarks plus ablations, qualitative results and a quantitative conflict analysis; however, the ablation covers only SD3.5-M, there is no cost analysis, and appendix details are absent from the main text.
Writing Quality: ⭐⭐⭐ The motivation and Figure 1 are persuasive, but several core reward equations are pushed to the appendix, symbol definitions are scattered, the commentary on Table 5 is duplicated in the main text (the same sentence names Table 5 twice), and the equations are visibly mis-typeset in the PDF.
Value: ⭐⭐⭐⭐ It offers a general, low-cost, pluggable weighting module for multi-reward RL post-training, with transfer value beyond compositional text-to-image generation.