AI-generated editorial illustration. It explains robot-computing placement and does not represent measured data.

1. The central question

A robot can carry its intelligence inside its body, send sensor data to a computer across the room, or call a model in a distant data center. Each choice changes what the machine can do—and how it can fail.

This is not merely an infrastructure decision. A manipulation policy may need fresh camera observations several times per second. A motor controller may need deterministic updates far faster than that. A language model planning a five-minute task can tolerate more delay, while an emergency stop cannot tolerate a network round trip at all. The location of computation therefore becomes part of the robot’s behavior.

The central conclusion is that there is no single correct location for “robot intelligence.” The practical unit of design is a layered control system: safety and fast feedback remain local; substantial perception and action models run on the robot or a nearby edge computer when possible; cloud services handle work that benefits from scale, shared knowledge or asynchronous coordination. The boundary should be chosen by deadline, failure consequence, data sensitivity and available power—not by model size alone.

2. Conceptual foundation

Inference is the process of running a trained model on new inputs. For a robot, those inputs may include images, joint positions, forces and language instructions; outputs may range from a task plan to a sequence of arm actions. The full path from sensing to actuation is a loop. Its delay includes sensor exposure, data transfer, preprocessing, model execution, command transmission and actuator response.

Average latency is only part of the problem. Jitter is variation in delay, and a rare long delay can be more dangerous than a modest but predictable one. A control component is real-time when it meets its deadline reliably, not simply when it runs quickly on average. ROS 2’s real-time guidance emphasizes avoiding nondeterministic operations such as page faults and dynamic memory allocation inside critical loops.

“On-device” means inference runs on compute physically carried by the robot. “Edge” means a nearby computer reachable over a local network, such as a server in a factory cell. “Cloud” means remote shared infrastructure reached over a wider network. These are not mutually exclusive products; they are locations to which different components can be assigned.

3. How the system works

A modern learned robot may contain several computational layers. At the bottom, embedded controllers read encoders and regulate current, torque, velocity or position. Above them, collision checks, state estimation and trajectory execution turn desired motion into safe actuator commands. A vision-language-action model may convert images and an instruction into short action chunks. A higher reasoning model may break “clear the table” into object-level steps, inspect progress and request a tool.

Each layer consumes the output of the one above while closing its own feedback loop. If high-level planning pauses, the local controller should still maintain balance, respect joint limits and stop safely. If a remote policy stream is interrupted, the robot needs an explicit fallback: finish a bounded motion, hold position, retreat or enter a safe state. Continuing the last command indefinitely is not a neutral default.

Deployment also has a data path. Camera frames may remain local, be compressed for an edge server, or be uploaded to the cloud. Model updates travel in the opposite direction. Logging may record sensor streams, decisions and interventions for debugging. Compute placement therefore determines not only response time but also bandwidth, observability, privacy exposure and the speed of fleet-wide updates.

4. Major approaches

Fully onboard inference

The robot carries the processors required for perception, policy inference and local planning. This suits mobile systems that must operate through outages, work around sensitive visual data or close action loops without network variability. Google DeepMind presents Gemini Robotics On-Device as a VLA optimized to run locally and specifically connects that design to latency-sensitive work and intermittent or absent connectivity.

The constraints are physical. More compute increases purchase cost, power draw, cooling demand and battery load. Model compression, quantization and optimized decoding can help, but may alter accuracy or flexibility. OpenVLA-OFT is an instructive research example: its fine-tuning recipe combines parallel decoding, action chunking and continuous actions to increase action-generation throughput, showing that software architecture can change the feasibility of local deployment.

Local edge inference

An edge server sits near the robot, usually on a controlled local network. It can host a larger accelerator, serve several robots and simplify maintenance without paying the full delay of the public internet. This works well in factories, warehouses and laboratories with engineered wireless or wired coverage.

The robot can be lighter and cooler, but the network becomes part of the control system. Congestion, handoffs and shared-server contention introduce jitter. Edge capacity must be budgeted for simultaneous demand, and each robot still needs local safety behavior when the server disappears.

Cloud inference

The cloud offers elastic accelerators, large models, centralized monitoring and rapid updates. It is suitable for asynchronous map processing, fleet analytics, knowledge retrieval and high-level planning whose deadline is measured in seconds rather than milliseconds. It can also let a product access a model too large to carry onboard.

The cost is dependency. Wide-area latency is variable, connectivity can vanish, and high-bandwidth sensor uploads create cost and privacy risks. Cloud inference is therefore a poor sole owner of balance, collision avoidance or immediate contact response.

Hierarchical hybrid inference

The hybrid architecture splits computation by timescale and consequence. A cloud or edge model proposes goals; an onboard VLA or motion policy produces bounded actions; local controllers enforce physical constraints. Google DeepMind’s description of Gemini Robotics 1.5 explicitly pairs high-level semantic reasoning with low-level safety subsystems onboard the robot. This layering is the most credible default because it allows powerful remote reasoning without making physical stability depend on the network.

5. Evidence and examples

Gemini Robotics On-Device provides a direct case for local VLA inference. DeepMind states that the model is optimized to operate on robotic devices, without a data network, and can be adapted with 50 to 100 demonstrations in its 2025 release. These are developer claims from the model maker, not an independent measurement of every hardware configuration, but they show that on-device general-purpose manipulation is an explicit model-design target.

OpenVLA-OFT shows the importance of inference optimization. Its authors report a 26-fold increase in action-generation throughput over their OpenVLA baseline while improving performance on LIBERO, using parallel decoding, action chunking and a continuous action representation. The comparison is specific to the paper’s implementation and benchmark; it should not be treated as a universal device speedup. It nevertheless demonstrates that deployment location cannot be decided from parameter count alone.

NVIDIA’s GR00T deployment documentation separates training servers from robot-side execution and describes Jetson AGX Thor as an edge platform for on-device inference. The company’s reference stack combines models, optimized runtime libraries and onboard compute. This is vendor documentation, but it exposes a real systems requirement: the model, runtime, memory capacity and thermal envelope must be engineered together.

Network research supplies the counterweight. NIST’s work on time-sensitive wireless networking for robotics measures latency, reliability and idle time rather than assuming a wireless link is deterministic. Its factory-automation program treats communications as part of the control architecture. That supports a general engineering rule: edge offload is credible only when network behavior is measured under representative load and the robot has a defined response to missed deadlines.

6. Trade-offs

Latency versus model capacity. Local compute reduces transport delay but may force a smaller or more compressed model. Remote compute can host a stronger model, yet its answer arrives through a variable network. The relevant comparison is end-to-end deadline performance, not accelerator speed in isolation.

Availability versus centralized updates. An onboard policy can continue through an outage. A centralized service can distribute improvements and revoke a faulty model quickly. A hybrid design can cache a validated local version while accepting remote updates only through a controlled process.

Privacy versus observability. Keeping images onboard limits exposure of homes, workplaces and people. Central logs make fleet debugging and incident investigation easier. Systems can send compact events, selected clips or derived features rather than unrestricted raw video, but those representations may still contain sensitive information.

Power versus bandwidth. Local accelerators consume energy and require cooling. Offloading saves some onboard computation but spends radio energy and network bandwidth. For a battery-powered robot, the balance depends on model duty cycle, sensor volume, connection quality and how long the robot must operate.

Safety versus flexibility. Remote models can use broad context and be updated rapidly. Local certified or constrained components are easier to bound. The safest division preserves local authority to reject, limit or stop remotely proposed actions.

7. Current limitations

Public model announcements rarely provide enough comparable measurements to select hardware directly. End-to-end latency, power, thermals, control frequency and performance after quantization are often reported under different robots and tasks. A model that “runs on-device” may still require an expensive high-power computer carried by a large platform.

Network tests can also mislead. A quiet laboratory Wi-Fi link does not represent a crowded factory, a moving outdoor robot or a home router during heavy traffic. Tail latency, packet loss, reconnection behavior and server contention need scenario-specific measurement.

Hybrid architectures add coordination failures. State can become stale between local and remote planners. Two layers may issue incompatible goals. Version mismatches can make logs difficult to interpret. The architecture needs timestamps, cancellation semantics, bounded command horizons and a clear owner for every actuator-level decision.

Finally, placing computation locally does not automatically make it safe or private. Onboard models can still produce unsafe actions, and stored sensor data can still leak. Location changes the threat and failure surface; it does not remove it.

8. Open questions

Which functions must never leave the robot? Joint stabilization and emergency stopping are obvious candidates, but the boundary for learned collision avoidance, grasp reflexes and whole-body balance is still system-dependent.

How should a robot switch between local and remote models without a behavioral discontinuity? A handover protocol must reconcile internal state, incomplete action chunks and different model capabilities. Evidence is needed on whether such switching can happen during manipulation rather than only between tasks.

How much capability is lost through compression? Parameter reduction, quantization and distillation may affect spatial reasoning, language grounding and fine manipulation differently. Deployment reports should pair efficiency measurements with task-specific performance after optimization.

Who controls updates and logs? Fleet learning benefits from centralized data, while owners may require local retention and auditable consent. Technical systems will need policies for what leaves the robot, how long it is stored and how a remote model change is rolled back.

9. Editorial synthesis

The strongest current design is not “edge first” or “cloud first.” It is deadline first. Assign every function a maximum tolerable delay, a response to missed deadlines and a consequence of failure. Then place it at the farthest computational layer that still satisfies those requirements with evidence.

Fast physical loops, safety constraints and outage behavior belong onboard. VLA inference can be onboard or at a managed edge depending on model efficiency, robot power and network quality. Cloud systems are most defensible for higher-level reasoning, shared knowledge, fleet operations and asynchronous learning. Whenever a remote service can influence motion, a local layer should retain the ability to constrain or reject it.

This architecture may look less elegant than a single giant model controlling everything. It is more credible because it treats intelligence as a system of interacting deadlines rather than a file placed on one processor.

10. Key takeaways

• Compute placement changes robot latency, jitter, resilience, privacy, power demand and failure modes.
• Average inference speed is insufficient; control systems need dependable end-to-end deadlines and safe behavior when a deadline is missed.
• Onboard inference favors autonomy and data locality, edge inference adds nearby shared capacity, and cloud inference favors scale and centralized services.
• The most credible architecture keeps safety and fast feedback local while distributing slower reasoning across onboard, edge and cloud resources.
• Model compression must be evaluated for both efficiency and loss of task capability.

11. Further reading

1. What Is a Vision-Language-Action Model?
2. Why Robots Generate Actions in Chunks: Autoregression, Diffusion and Flow Matching Explained
3. Beyond the Success Rate: How Generalist Robot Policies Should Be Evaluated

12. Sources & evidence

This article uses model-maker documentation, research papers, deployment documentation and public standards-oriented network research. Vendor claims are identified as such. Measurements from different systems are not treated as directly comparable. Full citations and evidence boundaries are recorded in sources.md and research-ledger.md.

Related reading

• Why Robots Generate Actions in Chunks
• What Is a Vision-Language-Action Model?
• Beyond the Success Rate

Sources

Primary and official sources

1. Gemini Robotics On-Device brings AI to local robotic devices
   Institution: Google DeepMind
   Published: 2025-06-24
   URL: https://deepmind.google/blog/gemini-robotics-on-device-brings-ai-to-local-robotic-devices/
   Accessed: 2026-09-02
   Supports: local VLA positioning, network independence, latency-sensitive use, trusted-tester SDK and 50–100 demonstration adaptation claim.

2. Gemini Robotics 1.5 brings AI agents into the physical world
   Institution: Google DeepMind
   Published: 2025-09-25
   URL: https://deepmind.google/blog/gemini-robotics-15-brings-ai-agents-into-the-physical-world/
   Accessed: 2026-09-02
   Supports: hierarchical use of high-level semantic reasoning and onboard low-level safety subsystems.

3. Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
   Authors: Moo Jin Kim, Chelsea Finn and Percy Liang
   Published: 2025-02-27
   URL: https://arxiv.org/abs/2502.19645
   Project: https://openvla-oft.github.io/
   Accessed: 2026-09-02
   Supports: OpenVLA-OFT recipe, parallel decoding, action chunking, continuous actions and the authors’ reported throughput and LIBERO results.

4. OpenVLA: An Open-Source Vision-Language-Action Model
   Authors: Moo Jin Kim et al.
   Published: 2024-06-13
   URL: https://arxiv.org/abs/2406.09246
   Accessed: 2026-09-02
   Supports: OpenVLA model scale, training-data scope and open VLA context.

5. Real-time programming in ROS 2
   Institution: Open Robotics / ROS 2 documentation
   URL: https://docs.ros.org/en/eloquent/Tutorials/Real-Time-Programming.html
   Accessed: 2026-09-02
   Supports: deadline-based real-time framing, jitter concerns and avoidance of nondeterministic operations in critical control loops. The cited page documents an end-of-life ROS 2 release but the systems principles are used only as background.

6. Isaac GR00T real-world deployment guide
   Institution: NVIDIA
   URL: https://github.com/NVIDIA/Isaac-GR00T/blob/main/getting_started/real_world_deployment.md
   Accessed: 2026-09-02
   Supports: separation of training infrastructure and robot-side deployment, direct local inference and Jetson AGX Thor deployment guidance.

7. NIST and Industry Researchers Make Advances in Wireless Time-Sensitive Networking
   Institution: National Institute of Standards and Technology
   Published: 2022-07-20
   URL: https://www.nist.gov/news-events/news/2022/07/nist-and-industry-researchers-make-advances-wireless-time-sensitive
   Accessed: 2026-09-02
   Supports: measurement of wireless latency, reliability and idle time in a robotics-related time-sensitive networking scenario.

8. Reliable, High Performance Wireless Systems for Factory Automation
   Institution: National Institute of Standards and Technology
   URL: https://www.nist.gov/programs-projects/reliable-high-performance-wireless-systems-factory-automation
   Accessed: 2026-09-02
   Supports: treating wireless communication as an engineered component of latency-sensitive industrial control systems.

Source and evidence notes

• Google and NVIDIA materials are first-party sources and are identified as model-maker or vendor claims.
• OpenVLA-OFT results are retained within the authors’ benchmark and implementation conditions.
• Results from different models or hardware platforms are not directly compared.
• The article’s architecture recommendation is editorial synthesis based on timing, connectivity, privacy and safety boundaries described in the sources.