HiPolicy: Hierarchical Multi-Frequency Action Chunking for Policy Learning¶
Conference: ECCV2026
Paper: https://eccv.ecva.net/virtual/2026/poster/5381
PDF: https://media.eventhosts.cc/Conferences/ECCV2026/pdfs/10308.pdf
Project: https://hipolicy.github.io
Area: Robotics & Embodied AI
Keywords: action chunking, multi-frequency modeling, imitation learning, diffusion policy, entropy-guided execution
TL;DR¶
HiPolicy makes a diffusion policy jointly predict low-frequency chunks covering longer horizons and high-frequency chunks for precise control, then adaptively selects an execution branch using sampled action entropy; the main table reports overall average success on selected RoboTwin tasks rising from 37% to 60% for DP and from 41% to 59% for DP3, but these aggregates cannot all be reproduced by simply averaging the visible task rows, and adaptive acceleration is evaluated separately.
Background & Motivation¶
Action chunking lets a robot predict a sequence of future actions rather than produce only the next control command from the current image. This helps reduce errors from stepwise decisions and represent continuous motion in demonstrations. Yet the number of predicted actions does not determine the temporal horizon by itself: with a fixed chunk length, denser sampling covers less elapsed time, whereas sparser sampling can span a complete manipulation phase but miss important adjustments near grasping, alignment, and contact. Diffusion Policy (DP) and 3D Diffusion Policy (DP3) model action distributions effectively, but a single temporal resolution still ties these competing requirements together.
The issue becomes especially apparent in demonstrations containing pauses. After pushing a microwave door almost closed, a robot may need to pause briefly before applying enough force to engage the latch. A very short observation history can make that pause look like task completion. Conversely, relying exclusively on sparse actions spanning longer intervals can remove the fine adjustments needed to grasp an edge or press a small button. The paper aims to preserve phase information in low-frequency histories alongside local control cues in high-frequency histories, rather than manually select one frequency for each task.
The hierarchy in HiPolicy therefore primarily concerns temporal sampling resolution, not an additional language planner or merely multi-scale visual processing. Core idea: organize observations and actions at matching frequencies, fuse short- and long-horizon information within one diffusion model, and use the concentration of its action distribution to select an execution frequency.
Method¶
Overall Architecture¶
The input consists of visual observation histories and optional proprioceptive states; vision can use images or point clouds, while proprioception includes joint and gripper information. Frequency-Aligned Conditioning prepares the inputs, Cross-Frequency Joint Generation produces action chunks at multiple temporal resolutions, and, when adaptive execution is enabled, repeated stochastic predictions for the same input feed Entropy-Guided Execution to select a branch for the robot.
Each branch contains the same number of action positions, but those positions correspond to different time intervals. A low-frequency branch can therefore describe a longer stretch of the demonstration, while a high-frequency branch retains fine changes between nearby actions. The branches exchange features during generation rather than act as unrelated policies. Execution uses only the selected branch, not an average of controls from different frequencies.
%%{init: {'flowchart': {'rankSpacing': 24, 'nodeSpacing': 28, 'padding': 6, 'wrappingWidth': 400}}}%%
flowchart TD
A["Visual histories and<br/>optional proprioception"] --> B["Frequency-Aligned Conditioning"]
B --> C["Cross-Frequency Joint Generation"]
C --> D["Entropy-Guided Execution"]
D --> E["Selected action chunk<br/>and robot execution"]
Key Designs¶
1. Frequency-Aligned Conditioning: give each action branch a history matching its time scale
Copying the same high-frequency visual features to every action branch does not provide genuine multi-scale temporal information. HiPolicy first samples observation frames at different frequencies from the raw history, then encodes each group; optional proprioceptive states pass through a multilayer perceptron. Low-frequency branches can access changes over a longer interval, while high-frequency branches receive densely sampled recent changes. The key is not an additional visual modality, but the temporal positions sampled for each observation group.
Hierarchical feature-wise linear modulation (FiLM) then injects each frequency's observation conditions into its corresponding action features. FiLM derives modulation parameters from the conditioning features to scale and shift action channels. The important correspondence is that low-frequency observations condition low-frequency actions and high-frequency observations condition high-frequency actions, rather than every branch sharing one fixed-frequency condition. In the ablation, using only high-frequency observations is more damaging than using only low-frequency observations, supporting the interpretation that additional action branches cannot substitute for longer-horizon observation context.
This alignment also prevents a mistaken reading of the hierarchy as a strict sequence in which a high-level planner finishes before a low-level tracker starts. HiPolicy does not first produce a complete low-frequency trajectory and then activate a separate high-frequency policy. Its branches receive conditions suited to their resolutions during joint generation. The hierarchy thus consists of concurrent temporal perspectives rather than a one-way command chain between two independent models.
2. Cross-Frequency Joint Generation: exchange local actions and overall phase information during denoising
Separate diffusion policies assigned different frequencies could each predict plausible actions while disagreeing about the current task phase. HiPolicy concatenates noisy action chunks from different frequencies along the temporal dimension and jointly predicts noise with a one-dimensional convolutional U-Net. Each branch retains local features, but a shared generation process handles all branches, allowing long-horizon intent and short-horizon corrections to inform each other.
Global feature fusion provides the actual cross-frequency communication. The paper uses cross-attention and a CLS token to aggregate action information from different frequencies, concatenates the resulting global feature with the local features at each frequency, and subsequently applies FiLM to incorporate observation conditions. Frequency-Aligned Conditioning determines which history an action level should consult; global fusion helps different levels describe the same manipulation phase. These roles are not interchangeable. For example, when the local image temporarily stops changing, longer-horizon action features can still indicate that the operation is ongoing rather than complete.
The output contains action chunks at several frequencies, not an upsampled low-frequency plan. The paper emphasizes integration with DP's image inputs and DP3's three-dimensional inputs, but this is not evidence for arbitrary policy architectures; experiments remain focused on those two diffusion baselines. Some tensor equations are damaged in the extracted full-text cache, so this note retains the concatenation, attention, and modulation relationships supported by the prose without inventing exact tensor dimensions or unavailable network settings.
3. Entropy-Guided Execution: interpret action flexibility as a phase signal rather than always slowing down
At execution time, HiPolicy independently samples multiple candidate action chunks for the same conditions. At a given frequency and action position, tightly clustered predictions indicate a narrow range of actions, whereas dispersed predictions may represent multiple feasible motions. The paper approximates the distribution at each position as Gaussian, measures its dispersion through differential entropy, and averages over frequencies and action positions to obtain the current chunk's entropy.
For a scalar Gaussian variable with standard deviation \(\sigma\), the relationship is:
Higher variance therefore produces higher entropy; this is differential entropy of a continuous distribution, not a success probability. Equation (13) uses variance and standard-deviation notation inconsistently, and the main text does not fully specify how multidimensional controls are reduced to the final scalar. The formula above only explains the Gaussian-entropy mechanism and is not used to reconstruct unspecified implementation details.
The selection direction is easy to misread. Low entropy selects high-frequency actions for fine corrections where the feasible action range is narrow, such as grasping. High entropy selects low-frequency actions to traverse permissive phases, such as approaching an object along multiple possible paths, with fewer executed steps. Preset ascending entropy thresholds correspond to descending frequencies; there is no additional gating network trained online. The algorithm retains the complete first sampled output for execution and uses the other candidates to estimate entropy. It neither selects the highest-scoring candidate nor averages all candidates.
This interpretation depends on an empirical association between demonstration action diversity and precision requirements. High entropy can also reflect out-of-distribution inputs or an unreliable model, so it does not generally imply that greater uncertainty should trigger acceleration. Moreover, low frequency describes the action sequence's sampling resolution, not necessarily a slower physical controller clock. The runtime comparison uses a fixed control frequency, with gains mainly coming from fewer executed control steps rather than faster model calls.
A Worked Example¶
Consider the microwave-door task discussed in the paper. Once the door is almost closed, a short history may contain nearly static images, causing a single high-frequency branch to stop. HiPolicy's low-frequency history still covers the preceding pushing motion, and Cross-Frequency Joint Generation can pass information about the unfinished latching phase to the fine-grained action branch. As execution continues, the entropy mechanism selects a resolution according to the concentration of the current candidate actions.
This is an explanatory walkthrough, not a record of intermediate variables released frame by frame by the paper. The reported task results are 0% success for DP and 100% for HiPolicy over 10 trials. HiPolicy takes an average of 58 steps to reach the almost-closed position and 67 steps to latch the door fully, so the first number alone cannot represent the complete task duration.
Loss & Training¶
HiPolicy follows Diffusion Policy's diffusion training procedure, conditionally predicting noise for concatenated multi-frequency noisy actions; the main text introduces no new reinforcement learning reward. The paper places the full training procedure and unified hyperparameters in Appendices C and D. The local file contains the complete main text and references but not those appendices, so learning rates, training epochs, branch frequencies, and exact entropy thresholds are not fabricated here.
The main text does specify parallel sampling with \(N=100\), extracting conditioning features once and reusing them. Under the reported hardware and batching setup, increasing the sample count from 1 to 100 changes latency from 105.5 ms to 107.5 ms and GPU memory from 7394 MB to 7457 MB. Shared conditioning and batching thus mitigate the extra sampling cost, but equivalent overhead is not guaranteed on devices with less memory.
Key Experimental Results¶
Main Results¶
Simulation uses 12 challenging tasks from RoboTwin 1.0 and 9 long-horizon or precise tasks from RoboTwin 2.0, rather than the complete benchmarks, with 100 evaluation episodes per task. Real-world experiments use a Franka Panda with a Robotiq 2F-85 gripper, two external cameras, and a wrist camera, evaluating 8 tasks with 10 trials each. Success rates below are higher-is-better, and gains are percentage points.
| Setting | Baseline | Baseline success ↑ | With HiPolicy ↑ | Gain |
|---|---|---|---|---|
| Selected RoboTwin tasks, paper-reported overall average (Table 1; see arithmetic caveat below) | DP | 37% | 60% | +23 |
| Selected RoboTwin tasks, paper-reported overall average (Table 1; see arithmetic caveat below) | DP3 | 41% | 59% | +18 |
| Precise-task subset (Table 1) | DP | 22% | 45% | +23 |
| Precise-task subset (Table 1) | DP3 | 28% | 53% | +25 |
| Real robot, 8-task average (Table 6) | DP | 60% | 85% | +25 |
Arithmetic caveat: The table above preserves the original overall averages from the paper's Table 1 rather than treating them as independently reproduced statistics. Equally averaging its 21 visible tasks gives approximately 37.57%, 59.95%, 41.43%, and 60.24% for DP, DP + HiPolicy, DP3, and DP3 + HiPolicy, respectively. Rounding to whole percentages yields 38%, 60%, 41%, and 60%, not the reported 37%, 60%, 41%, and 59%. Thus the first two rows' gains of +23 and +18 percentage points, and the paper's relative gains of 62% and 44%, are calculations based only on the printed aggregates, not overall improvements independently verified from task rows; relative gains must not be read as percentage points either. This note does not invent additional weights or execution settings to reconcile the discrepancy.
Execution settings also need to be separated: Table 1 does not explicitly state whether entropy guidance is enabled, and several of its task values match the “without EG” column in Table 3. Its 60% average therefore cannot simply be presented as the result with adaptive acceleration enabled.
Ablation Study¶
The following extracts two individual tasks from Table 2. Success is higher-is-better. Low-frequency-only and high-frequency-only conditioning retain multi-frequency action outputs and replace only the observation conditions.
| Config | Shoe place ↑ | Block hammer beat ↑ | Note |
|---|---|---|---|
| Full hierarchical model | 48% | 67% | Aligned conditions and cross-frequency fusion retained |
| Low-frequency conditions only | 43% | 66% | Longer history remains useful |
| High-frequency conditions only | 14% | 42% | Longer-horizon conditions missing |
| Without cross-frequency fusion | 43% | 56% | Weaker communication between branches |
| Without multi-frequency structure | 33% | 0% | Reverts to fixed-frequency action chunking |
Across Table 2's 12 tasks, the means are approximately 59.50%, 57.83%, 49.08%, 54.08%, and 36.58% in the order full model, low-frequency conditions only, high-frequency conditions only, without fusion, and without multi-frequency structure. Whole-percentage rounding matches the reported Average row of 60%, 58%, 49%, 54%, and 37%, so that table has no aggregate-mean contradiction. On the two tasks above, removing fusion costs 5 and 11 percentage points, respectively. Removing the multi-frequency structure costs 67 percentage points on Block hammer beat, but this does not imply comparable gains on every task.
Execution strategies are compared separately below using the 5 RoboTwin 2.0 tasks actually listed in Table 3. Its caption says “8 tasks,” conflicting with the visible rows and the runtime section's description of 5 tasks.
| Method | Average success ↑ | Average execution steps ↓ |
|---|---|---|
| DP | 24% | 133 |
| DP + HiPolicy, without EG | 45% | 139 |
| DP + HiPolicy, with EG | 41% | 100 |
Enabling EG saves 39 steps relative to disabling it, approximately 28.1% using the tabulated averages, while losing 4 percentage points of success. The roughly 25% step reduction highlighted in the paper's table instead uses DP's 133 steps as the baseline; these denominators must not be mixed.
Key Findings¶
- Acceleration does not mean faster inference. In Table 5's RTX 4090 measurements, DP and HiPolicy take 100 ms and 108 ms per inference and use 6560 MB and 7457 MB of GPU memory. Fewer execution steps reduce the reported rollout time from 10.57 s to 8.07 s. That time is computed from inference calls and a fixed 15 Hz control rate, not a platform-independent speed metric.
- More samples are not monotonically better. Table 4 reports 41% success at \(N=100\) and 39% at \(N=500\), with the latter increasing latency to 147.9 ms. This supports a moderate sample budget rather than unlimited candidates.
- Improvements are not universal across tasks. In Table 1, DP3 drops from 10% to 3% on Pick apple messy; average benefits in horizon modeling and fine control must not hide individual regressions.
Highlights & Insights¶
- Temporal scale affects both inputs and outputs. Multi-frequency actions need matching observation histories to exploit phase information rather than merely generate control points at different intervals.
- Joint generation and execution selection are distinct. The former shares context across resolutions, while the latter determines which branch actually runs; the ablations also show that improving success and reducing steps are separate outcomes.
- Action entropy acts as a task-phase cue. This reuses distributional information already available in a generative policy, but its interpretation depends on contact precision and demonstration diversity rather than a universal uncertainty-control rule.
Limitations & Future Work¶
- The authors explicitly acknowledge evaluating relatively small models and datasets without integration into large vision-language-action models. DP/DP3 results should not be extrapolated to general-purpose robot models.
- Simulation tasks are selected, real-world tasks have only 10 trials each, and sufficient confidence intervals are not provided to establish the stability of small differences. Entropy-gating reliability under substantial environmental change remains insufficiently tested.
- A Gaussian approximation can obscure multimodal action structure, and the transferability of fixed thresholds needs clearer evidence. Missing threshold and multidimensional entropy-aggregation details in the local main text limit direct reproduction.
- Table 1's overall averages cannot all be reproduced by equally averaging the visible task rows, and Table 3's task count conflicts with its caption. Further verification should first clarify the origin of the main-table totals, the EG setting, and the acceleration task set before interpreting aggregate gains; Table 2's averages are explained by ordinary rounding.
Related Work & Insights¶
- vs Diffusion Policy / DP3: These are the two-dimensional and three-dimensional diffusion policies directly extended and compared in this work. HiPolicy adds temporal hierarchy in observations and actions plus cross-frequency communication, rather than replacing the entire visual representation.
- vs H3DP: The paper describes H3DP as strengthening observation–action coupling through multi-scale visual representations. HiPolicy instead focuses on histories and futures associated with different sampling frequencies, not treating visual resolution itself as an action-time hierarchy.
- vs Reactive Diffusion Policy: That method separates slow high-level diffusion from fast visual-tactile control, whereas HiPolicy jointly generates multi-frequency actions and selects a branch through sampling entropy. No direct experimental comparison is provided, so architectural differences do not establish a performance ranking.
Rating¶
- Novelty: 4/5. Observation–action frequency alignment and joint generation are clearly motivated, with a task-specific entropy selection rule.
- Experimental Thoroughness: 3/5. Covers image inputs, three-dimensional inputs, and real robots, but the baseline range is limited and statistical and execution settings are not consistently described.
- Writing Quality: 3/5. The method's main narrative is clear, while equation notation, Table 1's overall averages, and acceleration denominators need closer checking.
- Value: 4/5. Offers a useful temporal-hierarchy design for diffusion action chunking, subject to verifying control and statistical details before deployment.