Bear’s robots already navigate and coordinate work at customer sites around the world. KINO-2 brings that same production discipline to manipulation. It is the learning and deployment platform behind KR1, Bear’s humanoid robot, and it is already at work in customer pilots, connecting object handling with the mobility and workflow systems needed to finish a real task.
Most robot learning stops at the lab demo. KINO-2 is built for what comes next: running learned skills reliably on a physical robot, inside a customer’s workflow. It builds on the data and learning infrastructure behind Bear’s mobile robot fleet and extends it with manipulation demonstrations and feedback from customer pilots. This paper explains that architecture and engineering results behind it.
1 From robot learning to real-world workflows
Bear has more than 16,000 robots in commercial service across many different environments, from hospitality to commercial facilities and warehouses. Running at that scale has shaped how we build navigation and coordinate fleets, and it gave us the data and learning infrastructure we now use to improve robot perception.
KINO-1 established our in-house approach to learning manipulation from demonstrations. KINO-2 extends that work for commercial deployment in two ways: a common framework for training and deploying different model architectures, and an accelerated runtime that runs more capable models onboard. This lets us scale our own models and bring in post-trained foundation models such as NVIDIA Isaac GR00T as customer workflow demands grow.
A customer workflow rarely ends when a robot reaches its destination. Goods may need to be picked, packed or replenished using equipment already on site. KINO-2 teaches these skills through demonstrations and simulated practice, cutting the custom programming each new workflow would otherwise need.
Take a warehouse packing workflow. Learned manipulation picks bottles and fills crates, while Bear’s mobility and workflow software handles crate transport and the sequence of work. Customers need the complete task done, not a single skill, so the complete task is what we build and measure.
Figure 1. KINO-2 connects learning and evaluation with onboard execution and Bear’s existing robot systems
This shared infrastructure ties what happens at a customer site directly to how a skill is developed and deployed. Models connect to KR1 through common robot interfaces, so a new model architecture reuses the mobility and workflow systems already running in the field. Model selection follows the task’s requirements and the robot’s onboard constraints.
2 Teaching robots to learn and adapt
Real work is unpredictable. A bottle can shift during contact, and a grasp can finish in a different position than expected. KINO-2 responds through a repeated observe, predict and act cycle: a model uses the current observations to predict a sequence of actions, and the robot executes part of that sequence before replanning. The model implements a policy, the mapping from observations and task context to robot actions. The diagrams below illustrate this approach using our in-house manipulation model.
Combining observations and task context
Figure 2. Bear’s in-house manipulation architecture
The same scene can support several valid actions. Task conditioning tells the model which behavior is required; spatial cues can identify the object and its destination. In a crowded packing area, this helps distinguish the bottle to pick and the position where it belongs.
The architecture also includes planned conditioning paths, marked in the diagrams. A segmentation-mask pathway would identify relevant image regions. The conditioning inputs and their encoding depend on the model architecture.
Predicting and updating actions
The action expert uses encoded observations and task context to predict a short sequence of arm and gripper movements. Predicting ahead gives the motion a horizon; replanning from fresh observations lets the next prediction account for what the robot actually did.
Figure 3. Configuration of Bear’s in-house manipulation model
The illustrated in-house configuration predicts 40 action steps at 10 Hz, representing a four-second horizon. KR1 executes one 0.1-second step before replanning from updated observations. This lets subsequent predictions respond to changes such as a bottle shifting during contact. The observation history and execution horizon are specific to this model configuration.
Supporting larger model architectures
KINO-2 supports both single-task and multi-task models within the same learning and deployment infrastructure, including, our in-house models and post-trained foundation models. We match the model to the workflow’s required behavior and onboard compute, introducing larger architectures as customers need broader capabilities.
Supporting different models requires more than replacing a checkpoint. Each retains its own encoders and action-generation architecture. We adapt its data representation and robot interfaces, keeping the meaning of observations and actions consistent from training through execution.
That flexibility depends on two foundations: task data that captures real-world variation, and an onboard runtime that turns predictions into timely robot actions. The next two sections cover each.
3 Scaling robot learning through data and simulation
A skill that only works on its recorded example is not useful to a customer. We start with demonstrations of the customer’s actual task, then broaden that experience through visual augmentation and simulated interaction. Each method plays a distinct role within the same learning pipeline.
Starting with real task demonstrations
Demonstrations capture how the work is done. Through teleoperation and other supervised workflows, we record camera observations alongside robot state and operator actions. Synchronizing these records preserves the relationship between what the robot sees and how it moves, including the timing of grasp and release. The recordings supply training and post-training examples for the selected model.
Broadening visual variation with synthetic data
A task can look different under new lighting or against a different background. Our augmentation pipeline uses NVIDIA Cosmos open world models to restyle demonstration videos while retaining their associated action, state and timing data. The purpose is to vary appearance while preserving the demonstrated motion so that the action labels remain usable. This paper describes the data-generation capability; it does not quantify a task-performance gain from Cosmos augmentation.
Parallel training and testing in simulation
Simulation lets a model learn through interaction, including attempts that do not succeed. Parallel environments make it practical to vary task setups, repeat difficult cases and develop recovery behavior under controlled conditions.
We train KR1 policies in NVIDIA Isaac Lab, an open training and evaluation framework, running 1,024 simulated KR1 instances in parallel in the pick-task configuration. That gives the learning process over a thousand copies of the task at once, without requiring a thousand physical robots.
In one simulated pick-task training run, time per pick halved, from 7.4 to 3.7 seconds, as training progressed. The model used simulator-provided states as input, so the result shows improvement within that training environment; physical-robot tests add camera observations, contact and real execution timing.
These data sources feed a one development loop.. Simulation provides controlled practice at scale, while hardware tests and customer pilots surface the real-world conditions that the next training cycle needs to address.
Figure 4. Parallel simulated task episodes from Bear’s bottle-handling data workflow.
Figure 5a. KR1 performing a bottle-handling task in a simulated warehouse.
Figure 5b. Simulated bottle-handling scenes with varied workspaces and visual conditions.
Figure 5c. Parallel KR1 training in NVIDIA Isaac Lab.
4 Accelerating KINO-2 with NVIDIA Isaac OS
A larger model is useful on a robot only if the complete system can keep up. On KR1, learned manipulation shares an NVIDIA Jetson AGX Thor with the robot’s existing autonomy software, so every millisecond counts. NVIDIA Isaac OS, an open-source, deterministic runtime, forms the backbone of our perception runtime. Bear’s engineering around it produced the most significant results in this paper: 4.2× faster inference, Isaac GR00T open VLA model running on the robot and substantially smoother motion, making efficient use of onboard compute from camera input through model inference.
An accelerated perception and inference pipeline
Isaac OS is built on NVIDIA Holoscan SDK (HSDK) 5.0. We use HSDK to connect camera ingestion to observation processing and inference in an accelerated GPU pipeline. Frames remain in GPU memory through preprocessing and model execution, while NVIDIA TensorRT optimizes execution of the model graphs.
Integrating the runtime with KR1
We custom-designed KR1’s Thor carrier board around this pipeline, replacing an earlier FPGA-based camera-board approach with an integrated design. Owning both the hardware and software lets us optimize the sensor-to-model path as one system. Early access to NVIDIA technologies and direct engagement with its engineering teams supported rapid iteration.
NVIDIA Isaac Bridge connects the Isaac OS/HSDK pipeline to our existing robotics stack, so the accelerated runtime works with the mobility and workflow KR1 already uses. Proven robot capabilities stay in place while allowing new models and their predicted actions to plug into the same task and control interfaces.
Accelerating the in-house model
Moving our in-house model into the HSDK 5.0 pipeline reduced reported inference time from 72.3 to 17.2 milliseconds—a 4.2× speedup over PyTorch eager execution. That headroom is what lets KR1 run larger post-trained foundation models on the same onboard hardware.
Running GR00T on Thor
Our optimized GR00T 1.7 model deployment runs at 56.6 milliseconds of median GPU execution time per action chunk on Jetson AGX Thor, down from 92.2 milliseconds in our earlier four-engine build. This combines the Isaac OS/HSDK pipeline with TensorRT engine optimization and selective FP8 quantization. The measurement covers model execution on the GPU.
We partitioned the model into eight TensorRT engines, applying FP8 to the vision transformer, diffusion transformer and cross-attention key/value projections. The largest gains came from the vision encoder and action head. We evaluate the resulting robot behavior alongside execution time as precision and engine configuration change.
From camera input to robot response
Fast model inference only matters if the robot responds in time. We measure the integrated path from camera exposure to joint motion, including observation processing and delivery of predicted actions to the controller, so our numbers reflect what the robot physically does.
Figure 8. Camera-to-motion response on Jetson AGX Thor, measured across the integrated sensing, inference and control path.
Optimizing the execution path
We also improved the path from observations to controller commands. Exposure-aligned camera pairing associates images with the relevant action sequence. Removing startup delay and refining action-chunk execution complement the accelerated inference pipeline.
Alongside faster inference, these execution changes cut commanded jerk by about 7×, from 237 to 33 m/s³, and improved command tracking, so the controller follows the intended action sequence more smoothly.
Figure 9. Commanded jerk on the hardware rig before and after execution changes. Jerk is the rate of change of commanded acceleration, measured in m/s³.
Together, these results show a system that processes observations and delivers motion commands fast enough for real work. The next question is how that performance holds up inside a customer’s task.
5 From engineering progress to customer deployment
KINO-2 is already at work in customer environments. Active customer proofs of concept (PoCs) span warehouse crate handling, supermarket returns and industrial tasks. Each pilot places KR1 inside a real workflow and sets the operating context for demonstrations and task evaluation.
Bottle packing shows how this works in practice. We examine where the model predicts a bottle will be released, when the gripper opens and whether successive predictions remain consistent. Comparing these predictions with demonstrations ties model behavior directly to the customer’s packing task.
Every pilot makes the next deployment easier. KINO-2’s model-agnostic learning infrastructure feeds customer experience into a shared evaluation process. Task outcomes and human interventions reveal where a skill needs improvement, and we combine that evidence with offline analysis and tests on KR1 to decide whether to collect targeted demonstrations, refine a model or change how its actions are executed. Those operating insights carry forward as the underlying models evolve.
Conclusion
KINO-2 brings the learning and operating infrastructure behind Bear’s commercial robot fleet to work that requires complex manipulation. The results in this paper show the engineering foundation is in place: faster onboard inference, foundation models running on the robot and smoother commanded motion on KR1. Bear is building toward a coordinated fleet that can pack a crate and deliver it as one continuous workflow, where every customer site makes the next deployment faster to stand up.
