LightParkour · Whole-body intelligence ·
Growing Humanoid Parkour Skills through Real2Sim2Real
LightParkour uses humanoid parkour as a testbed for contact-rich skill learning. It grounds short human-motion clips in physics, grows each seed across terrain variation, and distills locomotion and whole-body skills into one deployable depth policy.
One policy across locomotion and parkour. A single onboard policy drives both locomotion and contact-rich parkour on Lightbot 0.
The real data problem
A motion without its obstacle is only half a demonstration.
Parkour makes the central difficulty of contact-rich behavior explicit: the environment becomes part of the robot’s support structure. Success depends on where contact lands, when it bears load, and how the motion adapts as geometry changes.
Motion-tracking policies reproduce expressive whole-body skills but remain tied to the demonstrated scene. Reward-driven locomotion adapts to terrain, yet tends toward leg-dominant behavior. Neither alone produces geometry-aware, load-bearing contact.
Motion capture and ordinary video preserve movement, but rarely the scene that made it possible. Direct retargeting can leave hands floating, limbs intersecting obstacles, or trajectories the robot cannot execute.
Collecting a new demonstration for every obstacle restores the correspondence, but scales linearly with skill, contact pattern, and terrain. LightParkour instead recovers one short seed per skill and lets simulation generate the variation.
Real2Sim2Real
Simulation rebuilds the interaction, then expands it across terrain.
Real2Sim2Real turns each seed into a deployable skill family: physics restores valid contact, successful rollouts expand the terrain range, and distillation produces one recurrent depth policy.
-
Real
Recover interaction intent
Whole-body timing and intended contacts from the source clip.
-
Sim
Ground it in physics
Obstacle geometry, contact forces, and the robot’s actuator limits—then expansion across a range of obstacles.
-
Real
Deploy from perception
One depth policy acts without a runtime motion reference.
Speed vault. Human motion → physics-grounded reference → onboard execution.
Reverse vault. Human motion → physics-grounded reference → onboard execution.
Physics repairs what retargeting breaks.
We retarget the source motion and place the obstacle where it implies load-bearing contact. The initial pair is physically invalid: hands sink, knees clip, and human dynamics exceed the robot’s limits. A tracking policy repairs the interaction under full physics, anchoring intended contacts and restarting from random reference frames to expose critical phases. The maneuver is preserved; the trajectory becomes executable.
Iterative reference refinement. From an obstacle-misaligned seed to terrain-paired, dynamically feasible references.
Each successful rollout raises the obstacle.
Each successful rollout becomes the next reference, feasible by construction under full physics and actuator limits. We raise the motion and obstacle by 5–10 cm and train again. Iteration grows one 45 cm climb into a continuous family through 75 cm—83% of the 90 cm robot’s standing height.
Only the initial motion–obstacle pair is aligned manually; physics and curriculum drive the remaining expansion.
The family compresses into one policy.
A motion seed specifies interaction intent, not a deployable trajectory. Simulation builds a family of feasible contact strategies. Privileged teachers learn those families; multi-expert and recurrent depth distillation compress them into one onboard policy.
LightParkour / system
The whole loop, stage by stage.
Grow one seed into a skill family.
Locomotion + three contact-rich skills track the references under full physics.
Success becomes the next reference; the obstacle rises 5–10 cm.
Unify skills. Learn the handoff.
Multi-expert DAgger
Locomotion and whole-body skills share one action space.
Compress privileged geometry into depth.
Depth image · proprioception · command
The complete LightParkour training system. Reference generation grows terrain-paired references; multi-expert distillation unifies the skills; reward-only transition training learns the handoff; recurrent depth distillation produces a single 50 Hz onboard policy.
Multi-expert distillation consolidates the experts’ behaviors into one policy. Reward-only transition training then teaches it when to leave locomotion, enter a whole-body skill, and return. At deployment, depth and the velocity command drive those decisions—without a skill label or hand-authored switch.
During a vault, the chest camera may lose sight of the obstacle just as contact becomes load-bearing. Height-scan reconstruction trains the recurrent state to carry recent geometry through the occlusion. Training also models camera noise, dropout, 27–33 Hz frame rates, and 30–60 ms latency.
What proves it works
The policy has to survive four tests.
The evaluations test four requirements of a deployable contact-rich skill: physical feasibility, adaptation to geometry, composition across contact modes, and execution from onboard perception. All quantitative results are from simulation; hardware videos show transfer without real-world fine-tuning.
Operating range · Ablation
Fine-tuning and recurrence extend the deployable policy’s operating range.
Climb-and-step success across four obstacle heights, with 500 randomized simulation trials per setting.
- Full LightParkour
- No final task fine-tuning
- No recurrent memory
- Privileged teacher · height scan
60 cm
0.66H
65 cm
0.72H
70 cm
0.77H
75 cm
0.83H · frontier
Autonomous composition · Ablation
Learned transitions turn isolated skills into continuous behavior.
Inside the handoff · Policy memory
Transition training opens a route between the two behaviors.
Multiple skills in one network do not create transitions by themselves. With training, the hidden state leaves locomotion, passes through transition states, and enters parkour when terrain demands it; without training, trajectories remain in locomotion. The view below links each state to a reconstructed pose. Read temporal paths and local neighborhoods—not global cluster distances.
A learned route through policy memory. GRU hidden-state trajectories during transition rollouts, colored by locomotion, transition, and parkour. Lines preserve rollout order; each selected point is grounded in its corresponding reconstructed pose. This qualitative view complements the 100-trial transition-success ablation above.
Sim → Real
On hardware, only the deployable policy remains.
Lightbot 0 is our custom 90 cm, 18.9 kg humanoid with 21 actuated joints. A single recurrent policy runs onboard at 50 Hz from a chest-mounted depth camera, proprioception, and a velocity command—with no runtime reference, skill label, external tracking, or offboard state estimation.
Autonomous handoff
The policy times the handoff itself.
On mixed courses, the policy decides when to remain in locomotion, enter a whole-body skill, and return—without a runtime skill label or hand-authored switch.
Skill-to-skill. A direct handoff between two whole-body skills.
Skill-to-locomotion. A direct return to commanded locomotion.
Stair ascent and descent. Lightbot 0 uses onboard depth and the same deployable policy as the whole-body skills.
The same policy handles terrain between obstacles. On sparse footholds, it places each foot from support geometry that may already have left the camera’s field of view, relying on recurrent memory.
Narrow plank bridge. Precise foot placement on a highly constrained support surface.
Stepping stones. Foothold selection from onboard depth under partial observability.
Beyond parkour
Parkour is the testbed. Contact-rich autonomy is the larger goal.
As an early test beyond parkour, we apply the same pipeline to a separate loco-manipulation policy. Trained with a rigid box, it also carries a compliant ball unseen during training—an encouraging but preliminary result across shape, mass, and contact dynamics.
Trained on the box, carrying the ball. No retraining for the new object.
Parkour is where we started, not where the method ends. The aim is a robot that can observe a contact-rich interaction once, practice across geometry and dynamics in simulation, and carry the resulting skill into the real world.
Paper
LightParkour
Versatile Perceptive Whole-Body Locomotion via Multi-Skill Distillation
LightParkour grows sparse contact-rich motion seeds into terrain-paired reference distributions, unifies locomotion and whole-body skills, and learns transitions from reward. The recurrent depth policy transfers to Lightbot 0 without real-world fine-tuning.
@misc{chen2026lightlocoparkour,
title = {
{Light-Loco-Parkour}: Versatile Perceptive
Whole-Body Locomotion via Multi-Skill Distillation
},
author = {
Chen, Hongming and
Li, Zhuoran and
Wang, Hongxi and
Hu, Jiangpeng and
Li, Ziliang and
Liu, Peize and
Zhao, QingRui and
Liu, Xuhao and
Pan, Liang and
Lyu, Ximin and
Ma, Yuntao and
Fan, Tingxiang
},
year = {2026},
url = {https://light-loco-parkour.github.io/}
}