AI-generated editorial illustration. It explains action-sequence generation and does not represent measured data.
1. The central question
A robot ultimately needs concrete commands: move a joint, change a gripper position, or apply a velocity. Why, then, do many modern learning systems predict a short sequence of future commands instead of only the next command?
The answer is that useful motion has structure across time. A hand approaching a cup should not reconsider an unrelated direction at every control tick. Predicting a block of actions can preserve the rhythm and geometry found in a human demonstration. Generative methods can also represent several valid ways to complete the same task. But every additional future step introduces a risk: the world may change before the robot executes it. The central engineering problem is therefore not “chunks versus feedback.” It is how to combine temporally coherent action generation with frequent observation and correction.
2. Conceptual foundation
A policy maps an observation—camera images, joint states, language instructions or tactile signals—to an action. In behavior cloning, the policy learns this mapping from demonstrations. Straight regression can average contradictory examples. If demonstrators pass an obstacle on both the left and right, an averaged prediction may point toward the obstacle rather than either valid route. This is a multimodal action distribution: more than one action pattern is plausible in the same apparent situation.
Time creates a second problem. Actions are correlated: the next wrist movement depends on the trajectory already underway. A model that predicts one step at a time can accumulate small errors, while a model that commits too far ahead may ignore new information. An action chunk is a predicted sequence of commands covering several future control steps. A prediction horizon is how far the policy forecasts; an execution horizon is how much of that forecast the robot uses before observing again and replanning. Those horizons need not be equal.
3. How the system works
A chunked policy begins with current observations. A visual encoder converts images into features, while robot state and language may be encoded separately. A sequence generator then produces candidate commands for several future instants. The controller executes only an initial portion, receives a fresh observation and asks the policy for a revised sequence. This is receding-horizon control: future actions provide continuity, but the near future is repeatedly replaced with a plan based on newer evidence.
Training and inference differ. During training, a model sees demonstrated observation–action sequences and learns their conditional structure. At inference, ACT generates a chunk through a latent-variable sequence model; Diffusion Policy starts from noise and iteratively denoises an action trajectory; a flow model integrates a learned vector field from a simple initial distribution toward a likely action sequence; an autoregressive VLA predicts action tokens one after another. All eventually yield commands that must be decoded, safety-checked and sent to a lower-level controller at the required rate.
AI-generated editorial illustration. It explains the observation-to-action control loop and does not represent measured data.
4. Major approaches
Action Chunking with Transformers. ACT predicts a coherent block using a transformer and a conditional latent representation. It is well suited to demonstration datasets in which local motion patterns matter, including coordinated bimanual tasks. Temporal ensembling can blend overlapping predictions, reducing abrupt transitions. Its behavior depends strongly on chunk length: short chunks resemble stepwise control, while long chunks emphasize imitation of demonstration trajectories.
Diffusion over actions. Diffusion Policy learns to reverse a process that adds noise to action sequences. At inference, it refines noisy candidate trajectories through multiple iterations, conditioned on the observation. The method is attractive when demonstrations contain multiple valid modes or actions occupy a high-dimensional space. Receding-horizon execution makes the result closed-loop, but iterative sampling consumes inference time.
Flow Matching. Flow-based policies also transform a simple distribution into an action distribution, but learn a continuous transport field rather than a diffusion noise-reversal process. π0 uses a flow-matching action expert attached to a pretrained vision-language backbone. ACTIONFLOW combines Flow Matching with spatial symmetry biases. Flow approaches target expressive continuous actions with fewer sampling steps, although speed depends on the model, numerical solver, hardware and chosen integration schedule.
Autoregressive action tokens. A transformer can treat robot actions like a sequence of discrete symbols. Simple per-joint, per-timestep binning is easy to integrate with language-model training, but high-frequency signals produce long, highly correlated token streams. FAST first moves action sequences into a frequency representation using a discrete cosine transform, then tokenizes the compressed coefficients. This preserves an autoregressive training objective while making temporally smooth motion more economical to represent.
AI-generated editorial illustration. It contrasts action-generation families and does not represent measured data.
5. Evidence and examples
ACT’s authors evaluated six fine-manipulation tasks on low-cost bimanual hardware, including battery insertion and opening a translucent condiment cup. They reported 80–90% success using roughly ten minutes of demonstrations per task. This is evidence that sequence modeling can support precision on that setup; it is not proof that the same horizon or architecture transfers unchanged to other robots.
The journal version of Diffusion Policy reports 15 tasks across four manipulation benchmarks, spanning simulation and real robots, and an average 46.9% improvement over the baselines included by the authors. Its experiments support the value of multimodal sequence generation and receding horizons under those protocols. π0 provides a different scale of evidence: a flow-based VLA trained on varied single-arm, dual-arm and mobile-manipulator data, then evaluated for zero-shot behavior, instruction following and fine-tuning. The datasets and protocols differ, so their headline results should not be ranked against Diffusion Policy’s aggregate number.
FAST isolates another bottleneck: representation. Its authors report that frequency-space tokenization enabled autoregressive VLA training on high-frequency dexterous data where standard binning struggled. In a π0-based setup trained on 10,000 hours of robot data, FAST matched the evaluated diffusion-VLA performance while reducing training time by up to fivefold. That is a training-efficiency result under the paper’s configuration, not a universal claim about runtime latency.
6. Trade-offs
The first trade-off is coherence versus responsiveness. Longer chunks can stabilize motion and reduce jitter, but an unexpected object movement makes more of the remaining plan obsolete. Executing only a short prefix improves feedback at the price of more frequent inference.
The second is distributional expressiveness versus computation. Diffusion and flow models can represent multiple plausible trajectories rather than average them. Diffusion commonly requires repeated denoising; Flow Matching may use fewer integration steps, but neither label alone determines actual latency. Network size, observation encoding, solver settings and accelerator hardware matter.
The third is continuous precision versus a shared token interface. Continuous generators naturally express small changes. Discrete tokens fit the established machinery of autoregressive multimodal models and can share an objective with language, but the tokenizer determines what detail survives. FAST demonstrates that compression can make this route more practical; compression can also hide brief, high-frequency events if its design is mismatched to the task.
Finally, greater model capacity does not remove control engineering. A learned chunk still passes through rate limits, collision checks and low-level servo loops. Predicting elegant trajectories is valuable only if the complete stack can execute and interrupt them safely.
AI-generated editorial illustration. It explains engineering trade-offs and does not represent measured data.
7. Current limitations
Published evaluations remain fragmented. Different papers use different robots, control frequencies, demonstration quality, task definitions and success criteria. A method that excels at tabletop manipulation may not suit locomotion, deformable objects or safety-critical contact. Aggregate success rates conceal whether failures arise from perception, action generation or the controller below it.
Chunked policies also inherit dataset bias. They can reproduce smooth expert motion when the current situation resembles training data, yet generate confidently coherent mistakes under distribution shift. Generative sampling does not automatically provide calibrated uncertainty. Latency reports are similarly incomplete unless they include observation encoding, sampling steps, hardware and end-to-end control frequency.
8. Open questions
Can a robot choose its action horizon dynamically—long while free space is predictable, then short near contact or humans? Can policies expose uncertainty about an entire trajectory rather than only produce diverse samples? Researchers also need evaluation protocols that separate the quality of a proposed action sequence from perception quality and low-level execution.
Another question is whether a universal action representation can span different bodies and rates without erasing important dynamics. FAST+ is evidence that a shared tokenizer can cover diverse trajectories, while π0 demonstrates a shared generative policy across several platforms. Neither establishes that one representation can preserve every contact-rich or high-bandwidth behavior without embodiment-specific adaptation.
9. Editorial synthesis
The strongest current idea is not a particular generator but a system pattern: predict enough future motion to capture temporal structure, execute only what remains trustworthy, then replan. ACT, Diffusion Policy, flow-based policies and compressed autoregressive tokens are different ways to populate that loop.
For small demonstration datasets and well-defined manipulation, direct chunking offers a clear, practical baseline. When demonstrations contain visibly different valid behaviors or require high-dimensional coordinated motion, diffusion or flow models offer a better representation of alternatives. Autoregressive tokens become compelling when robot action is meant to share a scalable transformer with language and vision, provided the tokenizer respects temporal structure. The next meaningful evidence will come from controlled comparisons that hold robot, data and evaluation constant while measuring task success, recovery, end-to-end latency and interruption behavior together.
10. Key takeaways
- Action chunks preserve temporal structure, but only receding-horizon execution keeps them responsive to new observations.
- Diffusion and Flow Matching model distributions over action sequences; they differ in objectives and sampling, not simply “slow” versus “fast.”
- Autoregressive policies depend on tokenization, and frequency-space compression can reduce the cost of high-rate motion sequences.
- Results across ACT, Diffusion Policy, π0 and FAST are not directly comparable because their robots, data and protocols differ.
- The practical unit of design is the full feedback loop: observation, sequence generation, partial execution, safety control and replanning.
11. Further reading
- How Robots Learn from Demonstrations — begin with the data and behavior-cloning foundation behind every approach discussed here.
- What Is a Robot Foundation Model? — continue with how multi-task policies combine vision, language and action.
- Vision-Language-Action Models Explained — examine the larger architecture in which flow-based or token-based action heads operate.
12. Sources & evidence
- Zhao, Kumar, Levine and Finn, Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware, 2023.
- Chi et al., Diffusion Policy: Visuomotor Policy Learning via Action Diffusion, first published online 2024.
- Black et al., π0: A Vision-Language-Action Flow Model for General Robot Control, 2024; revised 2026.
- Pertsch et al., FAST: Efficient Action Tokenization for Vision-Language-Action Models, 2025.
- Funk et al., ACTIONFLOW: Equivariant, Accurate, and Efficient Manipulation Policies with Flow Matching, CoRL 2024 Workshop.
Sources
Accessed: 2026-09-02
-
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
Tony Z. Zhao, Vikash Kumar, Sergey Levine and Chelsea Finn. Submitted 2023-04-23.
https://arxiv.org/abs/2304.13705
Supports: Action Chunking with Transformers, error accumulation motivation, six-task real-robot evidence and reported demonstration requirements. -
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake and Shuran Song. First published online 2024-10-11 in The International Journal of Robotics Research.
https://journals.sagepub.com/doi/10.1177/02783649241273668
Supports: conditional action denoising, multimodal distributions, receding-horizon control, prediction/execution horizons and the paper’s 15-task evaluation. -
π0: A Vision-Language-Action Flow Model for General Robot Control
Kevin Black et al. Submitted 2024-10-31; revised 2026-01-08; published at RSS 2025.
https://arxiv.org/abs/2410.24164
Supports: the combination of a pretrained VLM and a flow-matching action expert, multi-embodiment training and generalist-policy evidence. -
FAST: Efficient Action Tokenization for Vision-Language-Action Models
Karl Pertsch et al. Submitted 2025-01-16.
https://arxiv.org/abs/2501.09747
Supports: limits of per-step binning, discrete-cosine-transform compression, FAST+ training scale and paper-reported training efficiency. -
ACTIONFLOW: Equivariant, Accurate, and Efficient Manipulation Policies with Flow Matching
N. Funk, J. Urain, J. Carvalho, V. Prasad, G. Chalvatzaki and Jan Peters. CoRL 2024 Workshop.
https://www.dfki.de/en/web/research/projects-and-publications/publication/15714
Supports: flow-based action generation, SE(3)-aware inductive biases and feedback-control motivation.