Skip to content

Controlling Creativity in Concept Space: A Framework for Generative Models

Conference: EMNLP 2026
Paper: ACL Anthology
Code: https://github.com/LevinDana/concept_level_temp_code
Area: LLM Reasoning
Keywords: Concept-Level Temperature, Creative Control, Concept Graph, Surprise Allocation, Text-to-Image Generation

TL;DR

This paper introduces Concept-Level Temperature (Concept-\(\tau\)), a model-agnostic framework that identifies underspecified semantic attributes via structured concept graphs and maps a continuous creativity parameter into discrete surprise levels using convex renormalization functions, achieving monotonic linear, high-diversity creativity modulation without compromising input utility.

Background & Motivation

Modern generative systems, prominently spanning large language models and text-to-image diffusion pipelines, have exhibited remarkable fidelity and prompt-following capabilities, empowering users to synthesize intricate multimodal scenes from concise descriptions. Nevertheless, an elemental blind spot persists across existing interaction paradigms: while users can effortlessly specify what entities must inhabit the synthesized output, they possess no principled mechanism to dictate how much creativity the underlying generator should apply. Naive prompting heuristics—such as appending qualitative pleas like "be more creative" or introducing integer creativity scores—routinely collapse into a constrained subspace of stereotypical outputs or fail to elicit controllable, reproducible variation.

Standard operational mechanisms for stochasticity, including sampling temperature in autoregressive models and classifier-free guidance (CFG) scales in diffusion models, intervene exclusively at lower representational strata such as token logit distributions, Gaussian noise magnitudes, or gradient updates. Because these mechanisms operate below the semantic layer, their ripple effects on the compositional choices that define genuine creativity remain indirect, unpredictable, and volatile. Modest parameter shifts barely disturb conventional renderings, whereas surpassing delicate empirical thresholds precipitates catastrophic semantic drift, spatial incoherence, or garbled syntax. Furthermore, specialized end-to-end architectures that attempt to optimize creativity directly are fundamentally tethered to specific model weights, exhibit poor interpretability, and break down when subjected to complex, constraint-heavy prompts.

Grounded in classical cognitive theories of creativity, creative production is formalized as the deliberate synthesis of novelty and utility. Because human prompts are intrinsically underspecified, the unmentioned attributes of a scene naturally demarcate an expansive combinatorial space for exploration. Meaningful creative modulation must therefore intervene at the abstract concept level where semantic choices actually occur. Core idea: structure the underspecified semantic degrees of freedom into dynamic concept graphs, and introduce Concept-Level Temperature (Concept-\(\tau\)) to map a continuous parameter onto hierarchical discrete surprise tiers via convex allocation functions, compensating for perceptual diminishing returns while achieving monotonic linear, diverse, and utility-preserving creative control.

Method

Overall Architecture

The Concept-\(\tau\) framework shifts creative modulation into an explicit conceptual abstraction space. Its pipeline comprises three modular phases: concept graph construction, hierarchical surprise allocation, and breadth-first candidate sampling with iterative prompt augmentation. Given an initial underspecified prompt \(d\) and a desired continuous creativity parameter \(c \in [0, 1]\), the system extracts entity hierarchies and delineates utility versus novelty nodes, calculates an allocation vector across discrete surprise tiers, samples context-conditioned concept values top-down, and formats an enriched prompt \(d'\) for downstream generation.

%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
    In["Input underspecified prompt d<br/>and creativity parameter c ∈ [0, 1]"] --> G["Stage 1: Structured Concept Graph Construction<br/>Extract entity hierarchy; isolate utility from novelty nodes"]
    G --> A["Stage 2: Convex Renormalized Surprise Allocation<br/>Distribute surprise via Assign(n, k, c) to counter diminishing returns"]
    A --> S["Stage 3: Multi-Level Candidate Generation & Sampling<br/>Breadth-first traversal of G; populate candidates and compose d'"]
    S --> Out["Downstream Generative Model MG(d')<br/>Synthesize novel, diverse, and utility-faithful outputs"]

Key Designs

1. Structured Concept Graph Construction: Decoupling Utility Constraints from Novelty Exploration

To resolve the limitation where low-level sampling parameters indiscriminately perturb both essential scene requirements and free attributes, this design leverages large language model in-context reasoning to parse prompt \(d\) into an explicit hierarchical directed acyclic graph \(G=(V, E)\). Node set \(V\) is partitioned into utility concepts (concepts whose values are explicitly dictated by \(d\), such as "laptop state = closed" in "A cat sitting on a closed laptop") and novelty concepts (degrees of freedom left open, such as the cat's breed, fur pattern, pose, or background lighting). Utility nodes are rigidly locked to preserve semantic entailment (\(d' \rightarrow d\)), while novelty nodes serve as controllable exploratory variables. Edge set \(E\) encodes semantic dependency and ontological hierarchy (e.g., entity \(\rightarrow\) sub-attributes). This explicit graph formalization converts uncontrolled global generation into an interpretable set of discrete semantic decisions.

2. Convex Renormalized Surprise Allocation: Mitigating Cognitive Diminishing Returns

Human subjective perception of novelty adheres to the psychological law of diminishing marginal returns: injecting glowing hair into an otherwise mundane woodland scene produces an intense burst of perceived novelty, whereas the exact same attribute introduced into an already surreal scene with floating islands and burning trees registers minimal additional psychological surprise. To enforce a predictable, monotonic linear relationship between \(c\) and perceived novelty (Desideratum P1), the framework introduces a recursive top-down allocation algorithm \(\text{Assign}(n, k, c)\). For \(n\) active novelty concepts and \(k\) discrete surprise levels (\(l_1\) predictable, \(l_2\) plausible yet unusual, \(l_k\) out-of-the-box surreal), the count allocated to the dominant top tier is governed by a convex, surjective, non-decreasing helper function \(h_k(x): [0, 1] \rightarrow [0, 1]\) (empirically instantiated with \(h_3(x) = x^3\) and \(h_2(x) = x^2\)):

\[m = \lfloor h_k(c) \cdot n \rfloor\]

The algorithm subsequently calculates the invariant parameter interval \([c_{\text{left}}, c_{\text{right}}] = \{c' \mid \lfloor h_k(c') \cdot n \rfloor = m\}\), rescales the parameter linearly as \(c' = \frac{c - c_{\text{left}}}{c_{\text{right}} - c_{\text{left}}}\), and recursively populates the remaining \(k-1\) lower surprise tiers. The authors establish mathematical proofs demonstrating that \(\text{Assign}\) satisfies full combinatorial coverage of the allocation simplex and strict lexicographical monotonicity (\(\text{Assign}(n, k, c_1) \succ_L \text{Assign}(n, k, c_2)\) for any \(c_1 > c_2\)), using the steep convexity of higher-order polynomials to deliberately withhold extreme surprises until \(c\) reaches higher values, successfully linearizing perceived novelty.

3. Multi-Level Candidate Generation & Sampling: Context-Conditioned Semantic Instantiation

Addressing the reality that raw model self-information (token negative log-likelihood \(-\log P(x)\)) frequently exhibits pathological anti-correlation with visual commonsense surprise (e.g., language models assign higher surprisal to finding a common twig in a forest than to finding an anomalous grand piano or ship), the framework circumvents internal probability ranking via few-shot structured generation. Traversal of graph \(G\) proceeds in strict breadth-first order so that child concept candidates are explicitly conditioned on already sampled parent attributes. For concepts with bounded discrete domains (such as season), fixed candidates are mapped directly onto the \(k\) surprise tiers; for open-ended attributes, few-shot prompts generate candidate pools across all \(k\) discrete levels simultaneously. Concepts are assigned to levels according to the vector generated by \(\text{Assign}\), prioritizing high-capacity nodes for higher tiers, and sampled values are synthesized into a coherent, highly detailed prompt \(d'\).

Benchmark & Evaluation Protocol

Empirical validation was performed on 20 representative image descriptions randomly sampled from the COCO dataset, employing gemini-2.5-flash-image as the core image generator \(M_G\). The operational range of \(c\) was evaluated across four uniform levels: \(c_{\text{low}} = 0.0\), \(c_{\text{med}} = 0.33\), \(c_{\text{high}} = 0.66\), and \(c_{\text{ext}} = 1.0\). Concept-\(\tau\) was benchmarked against three distinct baselines: (1) Token-level Temperature (Token-\(\tau\)), adjusting sampling temperature \(t \in [0.1, 1.5]\) during prompt augmentation with Llama-3-8B-Instruct (capped at 1.5 to prevent utter degradation); (2) Structured Temperature (Structure-\(\tau\)), sampling masked slot completions across the identical concept graph templates using temperature-controlled decoding; and (3) Naive Prompting (Naïve-\(\tau\)), directly commanding GPT-5.2 with explicit numeric creativity ratings from 0 to 10.

Evaluation metrics combine human subjective perception and computational visual measures: 50 vetted annotators on Prolific completed 2,400 blinded ranking trials across a 1–4 creativity scale, yielding Pearson (\(\rho\)), Spearman (\(\rho_s\)), and Kendall-\(\tau\) (\(\tau\)) correlation coefficients against \(c\); visual surprise and diversity were quantified via cosine distances between CLIP image embeddings of the generated scenes; and utility was audited via three error rates measuring text contradictions (\(err_{\text{prompt}}\)), visual contradictions (\(err_{\text{image}}\)), and syntax collapse/gibberish (\(err_{\text{quality}}\)).

Key Experimental Results

Main Results

The table below presents the quantitative alignment with human judgment, CLIP visual distance metrics, intra-scene generation diversity, and semantic error rates across all evaluated paradigms.

Method User Study \(\tau\) User Study \(\rho\) Diversity (CLIP) CLIP Distance \(\tau\) CLIP Distance \(\rho\) \(err_{\text{prompt}}\) \(err_{\text{image}}\) \(err_{\text{quality}}\)
Concept-\(\tau\) (Ours) 0.75 ± 0.02 0.83 ± 0.02 0.30 ± 0.00 0.59 ± 0.03 0.74 ± 0.03 0.02 ± 0.01 0.07 ± 0.02 0.02 ± 0.01
Token-\(\tau\) 0.16 ± 0.04 0.17 ± 0.04 0.14 ± 0.00 0.14 ± 0.05 0.20 ± 0.06 0.03 ± 0.01 0.05 ± 0.01 0.93 ± 0.02
Structure-\(\tau\) 0.10 ± 0.04 0.11 ± 0.04 0.12 ± 0.00 0.01 ± 0.05 0.03 ± 0.06 0.00 ± 0.00 0.02 ± 0.01 0.72 ± 0.03
Naïve-\(\tau\) 0.66 ± 0.02 0.75 ± 0.02 0.21 ± 0.00 0.44 ± 0.04 0.64 ± 0.04 0.00 ± 0.00 0.01 ± 0.01 0.00 ± 0.00

Dynamic Novelty Range Coverage & Model Decoupling

To establish whether methods successfully span a wide, granular novelty spectrum without plateaus (Desideratum P2), the table below details average user ratings (scale 1–4) and CLIP cosine distances across the operational tiers (\(c_{\text{low}}, c_{\text{med}}, c_{\text{high}}, c_{\text{ext}}\)):

Method Human \(c_{\text{low}}\) Human \(c_{\text{med}}\) Human \(c_{\text{high}}\) Human \(c_{\text{ext}}\) CLIP Dist \(c_{\text{low}}\) CLIP Dist \(c_{\text{med}}\) CLIP Dist \(c_{\text{high}}\) CLIP Dist \(c_{\text{ext}}\)
Concept-\(\tau\) (Ours) 1.09 ± 0.03 2.33 ± 0.07 3.09 ± 0.07 3.77 ± 0.04 0.15 ± 0.01 0.25 ± 0.01 0.29 ± 0.01 0.41 ± 0.01
Token-\(\tau\) 1.31 ± 0.05 1.31 ± 0.05 1.47 ± 0.06 1.61 ± 0.06 0.14 ± 0.01 0.14 ± 0.01 0.14 ± 0.01 0.19 ± 0.01
Structure-\(\tau\) 1.32 ± 0.05 1.27 ± 0.05 1.42 ± 0.05 1.43 ± 0.06 0.16 ± 0.01 0.15 ± 0.01 0.15 ± 0.01 0.16 ± 0.01
Naïve-\(\tau\) 1.17 ± 0.04 1.21 ± 0.04 1.69 ± 0.06 3.60 ± 0.05 0.13 ± 0.01 0.12 ± 0.01 0.14 ± 0.01 0.36 ± 0.01

In an ablation decoupling algorithm mechanisms from proprietary model advantages, all methods were re-evaluated over the identical open-source backbone Qwen2.5-7B-Instruct (under \(k=2\)). Concept-\(\tau\) sustained strong correlation metrics (\(\tau = 0.64\), \(\rho = 0.80\)), whereas Token-\(\tau\) collapsed to \(\tau = 0.08, \rho = 0.13\), and Structure-\(\tau\) to \(\tau = 0.04, \rho = 0.07\), confirming that algorithmic concept-level allocation, rather than base LLM scale, drives the performance gains.

Key Findings

  • Monotonic Linearity across Subjective and Visual Regimes: Concept-\(\tau\) achieves remarkable correlation with human judgment (\(\rho = 0.83\)) and visual feature shift (\(\rho = 0.74\)), delivering progressive and predictable novelty gains. In contrast, token-level temperature baselines remain largely unresponsive across low and medium parameters, yielding stagnant user ratings near 1.3.
  • Pathological Step-Function Collapse of Naive Prompting: Direct numeric prompting (Naïve-\(\tau\)) displays severe mode collapse; user scores remain virtually flat between \(c_{\text{low}}\) (1.17) and \(c_{\text{high}}\) (1.69), before abruptly jumping to 3.60 at \(c_{\text{ext}}\). It completely lacks smooth intermediate exploration capabilities.
  • Robust Utility Preservation vs. Token Chaos: Escalating token temperature severely corrupts prompt syntax, with Token-\(\tau\) exhibiting a 93% degradation rate (\(err_{\text{quality}} = 0.93\)) characterized by repetitive tokens and omitted entities. Concept-\(\tau\) confines text errors to 2% (\(err_{\text{quality}} = 0.02\)) while preserving scene identity across 93% of generated visual outputs.

Highlights & Insights

  • Mathematically Compensating for Cognitive Perceptual Saturation: Rather than adopting naive linear concept scaling, the work uncovers the cognitive diminishing returns of unexpected stimuli and formulates non-linear convex mapping functions (\(x^2, x^3\)) within \(\text{Assign}\), successfully producing a perceptually linear external response curve.
  • Exposing the Multi-Modal Mismatch in Self-Information: The paper reveals that text-based token perplexity does not correspond to visual semantic surprise. Bypassing internal token likelihoods in favor of structured contextual few-shot sampling eliminates counter-intuitive candidate rankings.
  • Broad Model-Agnosticism and Independent Sub-Attribute Control: Operating entirely at inference time via prompt enrichment, Concept-\(\tau\) requires zero architectural modifications or retraining. Furthermore, because concepts are localized in a graph, it naturally enables spatially decomposed creativity (e.g., modulating high surrealism exclusively on a vehicle while keeping the surrounding landscape photorealistic).

Limitations & Future Work

  • Domain-Specific Adaptation Overhead: The framework is not entirely domain-free; extending it to alternate modalities (such as recipe creation, scriptwriting, or musical composition) requires crafting domain-specific few-shot examples and formalizing customized utility boundaries (e.g., verifying that culinary ingredients remain edible and cooking procedures physically feasible).
  • Residual Conceptual Mode Collapse at Extreme Tiers: Although concept-level sampling mitigates scene-wide mode collapse, LLMs occasionally gravitate toward recurring high-surprise tropes (such as repetitive depictions of stained glass, clockwork, or celestial nebulas) at \(c_{\text{ext}}\). Integrating external randomized knowledge retrieval could further broaden semantic entropy.
  • Computational Latency of Multi-Stage Candidate Probing: Executing iterative LLM calls to construct candidate pools across every underspecified node increases token overhead and inference delay. Distilling the allocation and sampling procedure into compact dedicated proxy models represents a promising optimization path.
  • vs ConceptLab & C3 (Feature-Level Amplification): ConceptLab expands single-category semantic boundaries, while C3 amplifies internal diffusion feature maps during denoising. These approaches are tightly coupled to specific diffusion architectures and struggle with multifaceted scene compositions. Concept-\(\tau\) is completely black-box and model-agnostic, functioning smoothly across complex multi-entity prompts.
  • vs POET (Automated Prompt Expansion): POET discovers unmentioned visual dimensions by reverse-clustering generated images; Concept-\(\tau\) achieves comparable concept discovery directly during the text forward pass, eliminating iterative image synthesis overhead while formally guaranteeing parameter monotonicity.
  • vs CrPO & Reinforcement Learning: CrPO optimizes creative generation by aligning models with preference rewards, requiring expensive policy retraining whenever control objectives change. Concept-\(\tau\) operates at test-time inference, offering real-time, continuous, and locally decomposed parameter control without parameter updates.

Rating

  • Novelty: ⭐⭐⭐⭐⭐ Formulates the first model-agnostic framework for concept-space creativity control with convex renormalization to correct perceptual diminishing returns.
  • Experimental Thoroughness: ⭐⭐⭐⭐⭐ Rigorously validated with 2,400 blinded human annotations, multi-scale CLIP geometric distances, and extensive error auditing.
  • Writing Quality: ⭐⭐⭐⭐⭐ Pristine mathematical formulations, compelling narrative flow, and comprehensive empirical validation.
  • Value: ⭐⭐⭐⭐⭐ Solves the longstanding challenge of quantitative, interpretable creativity modulation in generative AI, establishing a versatile paradigm for human-AI co-creativity.