Sim-to-Real of locomotion policies is usually framed as a physics gap traditionally in robotics. The industry responds with better contact models, friction cones, more domain randomization, etc. We benchmark simulators like Isaac Sim (PhysX) and MJX against each other (Sim2Sim) to make sure the math agrees.

Yet robots still fail in real life, in non-obvious ways.

Not because friction was 3% off, but because a CAN packet arrived late, a thread missed its deadline, or an IMU integration drifted just enough to destabilize the controller.

Subscribe now

Reality is noisy, late, and unfair. So how do we optimize for real world uncertainty?

In our latest work on Asimov Legs, bridged Sim2Real locomotion further by running our real robot firmware and control stack inside a deliberately imperfect simulated environment.

In this post, we share the methodology and results. We are optimistic about what it implies for hardware-software-AI co-design in modern robotics.

Don’t just simulate the robot, simulate the embedded environment

The core idea is to make the processor part of the robot’s dynamics in training.

We call this Processor-in-the-Loop because the control loop closes through the actual firmware execution model. We account for its threads, timing, and numeric constraints. This makes the processor itself part of the robot’s dynamics during training.

How it works:

• Sensor Fidelity: We don’t cheat. We don’t pass “truth” orientation to the controller. We stream raw accelerometer and gyro data from the MuJoCo lsm6dsox model over our I2C Emulator. This mimics the exact register-level behavior of the sensor bus.

• The Fusion Stack: The firmware runs the open-source xioTechnologies/Fusion C library to calculate projected gravity ($g_z$) in real-time. This is critical because Python math (infinite precision) and Embedded C math (constrained floats) often disagree. If our integration of the Fusion library drifts, or if the thread priority causes a bottleneck, the robot falls in real life.

• Binary Confidence: We aren’t running a simplified “simulation version” of our code. The firmware talks to our CANBus Emulator exactly as it would talk to the physical CAN hardware. It doesn’t know it’s in a simulation.

The implication of this pipeline is that we catch bugs in the pull request, not when the robot is standing up.

Account for Jitters and Delays

The biggest lie in robotics is the datasheet.

Our motor datasheet claims a 0.4ms response time. In a perfect simulation, that response is instant and constant. In the real world, it fluctuates. The “jittery time” domain needs to be optimized in training.

If you train a policy with Domain Randomization (specifically, randomized actuator delays), you believe your policy is robust. But you rarely test that belief until you are on the hardware.

Our Motor Emulator validates this explicitly.

The emulator sits between MuJoCo and the Firmware. It intercepts the Request-Response packets and injects a random delay drawn from a uniform distribution between 0.4ms and 2ms.

This turns “Domain Randomization” from a theory into an integration test. By forcing the firmware and policy to handle this jitter in CI, we catch specific failure modes that usually only appear in the field:

• Race conditions in the threading model (e.g., what happens if the control loop finishes before the motor response arrives?).

• Protocol parsing errors in the CAN driver under load.

• Jitter intolerance in the RL policy.

• Fusion library drift due to timing mismatches between sensor reads and control ticks.

Sim-to-Real Cannot be Solved in a Black Box

Previously, when we experimented with the Unitree G1, we couldn’t achieve these results on new polices. It was banging our heads against a black box. Closed firmware hides the dominant failure modes.

Sim-to-real is fundamentally an observability problem: you can’t fix what you can’t measure.

We deploy the policy on the Asimov robot equipped with 12 Series motors. The videos below show zero-shot sim-to-real locomotion behaviors using a single trained policy: forward and backward walking, lateral walking, and balance recovery under external pushes.

Sim-to-real rollouts of a Asimov’s locomotion policy. Left-to-right: forward walking(vx=0.6m/s), backward walking(vx=0.6m/s), side walking(vy=-0.6m/s), and push recovery (from human bully).

We also compared the joint position analysis graph in real(left) vs in sim(right), which let us directly check whether the policy’s motion patterns align across domains.

Deployment graph on the robot. Left: joint trajectories on real Asimov, Right: joint trajectories in sim.

Co-Design is the Point

Sim2Real fails because hardware, firmware, and policy are developed in isolation, then forced to agree at the very end.

The control policy needs to experience real timing and other frequent failure modes.

Our approach works because it treats the robot as one system.

1. Policies are trained assuming imperfect sensors and delayed actuators

2. Hardware is designed with policy capabilities and requirements in mind

3. Firmware is written to seamlessly close the gap

None of these layers are adapted later. They are designed together.

This is why we’re building Asimov as a robotic AI stack. A place where hardware, firmware, and models evolve in lockstep, and where Sim-to-Real stops being a gamble and starts being an engineering process.

The real world is chaotic and we should design every layer to survive it.

Share Asimov

Leave a comment

We’d love your feedback

Join our Discord here to contribute to our open source efforts and give us feedback.

Asimov is an open-source humanoid:

• Website

• Github

• Partnership

• Support

We trained a locomotion policy that got our humanoid robot Asimov to walk.

We focus less on algorithms and more on what information the policy is allowed to see, when, and why. We outline how we structured the observation space, why the critic has access to privileged signals that are intentionally hidden from the actor, how our reward design diverges from common open-source baselines like Booster Gym, and the lessons that emerged from running the policy on real hardware.

Subscribe now

Locomotion as a Data Interface Problem

Our locomotion policy is a standard feedforward network. What mattered was not the architecture, in this case a simple MLP, but the contract between the policy and the system it controls.

Early on, we observed that the robot oscillated violently on startup. We needed to understand why the policy learned to oscillate. After deep analysis, we realized the policy was behaving like an underdamped control system.

This led us to reframe the entire locomotion problem. We did not spend much hand-shaping the RL rewards around walking in simulation. Instead, we spent time investigating what data the policy needed to see during training, so it generalizes to real hardware.

Observation Space: Dropping Ground Truth Variables

Our policy observes data in 45 dimensions:

┌─────────────────────────────────────────────────────────┐
│ ACTOR OBSERVATIONS (45D)                                │
├─────────────────────────────────────────────────────────┤
│ base_ang_vel          (3D)  - IMU angular velocity      │
│ projected_gravity     (3D)  - Which way is down         │
│ command               (3D)  - vx, vy, ωz targets        │
│ joint_pos_group1-3   (12D)  - Joint positions           │
│ joint_vel_group1-3   (12D)  - Joint velocities          │
│ actions              (12D)  - Previous action output    │
└─────────────────────────────────────────────────────────┘

Notice what’s missing: base linear velocity (V_x, V_y, V_z).

Many open-source locomotion codebases feed ground-truth linear velocity to the policy. We actually didn’t do this. Why? Because on the real robot, you don’t have ground-truth velocity. You have an IMU that drifts, encoders that measure joint positions, and nothing else.

If you train with perfect velocity information, your policy learns to rely on it. Then it fails on hardware where that information doesn’t exist.

Staggered Observation Delays: Matching CAN Bus Timing

On a real robot, we read motor states over CAN bus. The motors are polled sequentially, not simultaneously. This means hip motors report data that’s 6-9ms stale by the time ankle motors report.

Most simulators ignore this. Every joint gets perfectly synchronized into zero-latency observations. The policy trains on pristine data, then fails when real data arrives out of order.

We modeled CANBus latencies explicitly:

# CAN read order creates timing skew
JOINT_GROUP_1 = ("hip_pitch", "hip_roll")      # Read first → 6-9ms stale
JOINT_GROUP_2 = ("hip_yaw", "knee")            # Read middle → 3-5ms stale
JOINT_GROUP_3 = ("ankle_pitch", "ankle_roll")  # Read last → 0-2ms fresh
# Each group gets different observation delay
"joint_pos_group1": delay_min=0, delay_max=2  # 0-10ms randomised
"joint_pos_group2": delay_min=0, delay_max=1  # 0-5ms randomised
"joint_pos_group3": delay_min=0, delay_max=0  # Fresh data

This is different from typical domain randomization. We’re not adding random noise for robustness. We’re intentionally modelling our training environment to match the TYPE and TIMING of data the policy will face in real life.

Asymmetric Actor-Critic: Teaching Through Privileged Information

During training, the actor only sees what the robot’s real sensors provide. The critic, however, has access to additional privileged information. This approach, called Asymmetric Actor-Critic, lets the policy learn to infer quantities it can’t directly observe.

What the Critic Sees

┌─────────────────────────────────────────────────────────┐
│ CRITIC OBSERVATIONS (Privileged Info)                   │
├─────────────────────────────────────────────────────────┤
│ Everything the actor sees, PLUS:                        │
│                                                         │
│ base_lin_vel          (3D)  - Ground truth velocity     │
│ foot_height           (2D)  - How high are the feet     │
│ foot_air_time         (2D)  - Time since last contact   │
│ foot_contact          (2D)  - Binary contact state      │
│ foot_contact_forces   (6D)  - Ground reaction forces    │
│ toe_joint_pos         (2D)  - Passive toe position      │
│ toe_joint_vel         (2D)  - Passive toe velocity      │
└─────────────────────────────────────────────────────────┘

The Critic Sees the Toes

Asimov has passive spring-loaded toes. They deflect under load but have no motor and no encoder, which means the robot doesn’t “sense” the toes. You can read more about this design decision here:

By exposing toe state to the critic, the policy learns the relationship between toe behavior and stability, inferring it from ankle positions and IMU readings during deployment.

# Critic sees toe state (privileged info)
"toe_joint_pos": ObservationTermCfg(
    func=mdp.joint_pos_rel,
    params={"asset_cfg": SceneEntityCfg("robot", joint_names=TOE_JOINTS)},
    scale=1.0,
)
"toe_joint_vel": ObservationTermCfg(
    func=mdp.joint_vel_rel,
    params={"asset_cfg": SceneEntityCfg("robot", joint_names=TOE_JOINTS)},
    scale=0.1,
)

The same principle applies to contact forces. The actor learns to “feel” the ground indirectly from joint positions and orientation. This improves stability and encourages more effective push-off while walking, even without direct force sensing.

Our current policy doesn’t instantaneously react to changes in ground contact force, like Unitree’s G1. Conversely, we observed quieter, more controlled footsteps when the robot was destabilized, a phenomenon we plan to study and improve further.

Improving Rewards

For rewards engineering, we were heavily inspired by Unitree, Booster Robotics, and MJLab’s existing open source rewards, and for the most part, kept the physics simple.

Of course, we made some deletions and additions per Asimov’s custom hardware. For example, we kept the following:

We made the following changes

1. No Gait Clock because all humanoids stride differently
Booster uses an explicit gait clock—sine/cosine phase signals that tell the policy “now is when the left foot should swing.”

# Booster's gait clock observation
cos(2π × gait_process),  # 1D
sin(2π × gait_process),  # 1D

We don’t use this. Our policy discovers its own gait frequency from the velocity command and body dynamics. This makes the policy more flexible but potentially less sample efficient.

Asimov has unusual kinematics (canted hips, backward-bending knees, limited ankle ROM). We wanted the policy to find gaits that work for this robot, not impose a human-designed gait structure.

2. Asymmetric Pose Tolerances because all humanoids have different flexibilities
Booster uses uniform pose tolerances across joints. We use asymmetric tolerances tuned to Asimov’s mechanical design:

std_walking = {
    "hip_pitch": 0.5,     # Large variance - canted hip design allows wide range
    "hip_roll": 0.25,     # Moderate - asymmetric joint limits
    "hip_yaw": 0.2,       # Standard
    "knee": 0.5,          # Large variance - knees extend backwards
    "ankle_pitch": 0.2,   # TIGHT - only ±20° ROM
    "ankle_roll": 0.12,   # TIGHT - only ±15° ROM
}

Asimov’s ankles have limited range of motion (hardware constraint from the parallel RSU mechanism). If we used large pose tolerances everywhere, the policy would try to use ankle ranges that don’t exist and fail on hardware.

3. Narrow Stance Penalties
Asimov has a narrower stance than most humanoids (~21cm stance width). This makes lateral stability harder. We increased penalties for:

"body_ang_vel": -0.08      # Pelvis rotation (was -0.05)
"angular_momentum": -0.03   # Global stability penalty

4. Contact Force Limits
We penalize excessive ground reaction forces:

This teaches the policy to place feet gently rather than stomping. Important for hardware longevity.

5. Air Time Reward
Asimov is light (~16kg legs). It can actually achieve flight phase during walking. We reward air time:

"air_time": +0.5

This encourages dynamic gaits rather than shuffling.

Domain Randomization: Targeted, Not Broad

We didn’t randomize everything, like mass, friction, gains, delays, and noise.
We took a more targeted approach towards robustness. We randomized the specific parameters that we knew differed between sim and real, based on our hardware characterization.

What We Randomise

What We intentionally did not randomize

• Body mass: Disabled for initial learning. Adding ±20% mass randomization made training unstable. We’ll add it in later for robustness fine-tuning.

• Link lengths: Our URDF matches the real robot accurately. There was no need to randomize geometry.

• Gravity: It’s 9.81 m/s² everywhere we deploy.

The philosophy for this initial policy was to randomise what you know varies, and don’t randomise what you’ve already measured accurately.

Network Architecture and PPO

Actor-Critic Networks

Actor:  obs(45) → 512 → 256 → 128 → actions(12)
Critic: obs(45) + privileged(~20) → 512 → 256 → 128 → value(1)
Activation: ELU
Observation normalisation: Enabled
Initial noise std: 1.0

Three hidden layers with decreasing width. ELU activations throughout. Nothing fancy.

PPO Hyperparameters

learning_rate = 1e-3          # Adaptive schedule
gamma = 0.99                  # Discount factor
lam = 0.95                    # GAE lambda
clip_param = 0.2              # PPO clip
entropy_coef = 0.01           # Exploration bonus
num_learning_epochs = 5       # Epochs per rollout
num_mini_batches = 4
desired_kl = 0.01             # Target KL for LR adaptation
max_grad_norm = 1.0           # Gradient clipping
num_steps_per_env = 24        # Rollout length

Compared to Booster (learning_rate=1e-5, mini_epochs=20), we use higher learning rate and fewer epochs. Our adaptive schedule reduces LR when KL divergence exceeds target.

What We’re Working on Next

• Terrain Curriculum: Currently training on flat ground. Next step is procedural terrain with curriculum learning, start flat, progressively add slopes, stairs, and rough terrain as the policy improves.

• Velocity Curriculum: Our current velocity range is conservative (±0.8 m/s forward, ±0.6 m/s lateral). We’ll expand this with curriculum learning once flat-ground performance is solid.

• Full Body Integration: The legs work. Now we’re integrating with arms and torso for the complete 26-DOF+ humanoid.

You can read more about how we designed the hardware and firmware in other posts in this series:

Menlo Research is building and open sourcing an autonomous Humanoid from scratch. It has a modular design, with the legs, arms, and head development happening in parallel. We build in public and share our wins (and failures on most days) openly. You can also follow us on X, Instagram, and YouTube.

Leave a comment

We built a full set of humanoid legs from scratch. Got them to walk. All in under 100 days. For under $30k in R&D. And we’re open sourcing everything.

Legs are the foundation of the locomotion problem in robotics. The mechanical design of our Asimov1 humanoid leg is the ultimate balancing act.

Designing for modularity

Asimov has a modular design. It consists of separate components, legs, torso, arms, head, that all snap together.

Our modular approach uses universal motor mounting fixtures, so components are swappable and the platform can be configured for different applications.

We did this for two reasons:

1. Different research labs have different priorities. Some teams want to focus on manipulation problems, others on locomotion, others on the “brain” or head unit. A modular design lets them work independently with the subsystems that matter to them.2

2. There’s also a practical reason. Modular designs enables assembly. Each part of the robot requires different motors. The motor that attaches an arm is completely different from what you need at the hip. Clear load paths, repeatable interfaces, and serviceable modules mean the robot can be built and maintained without specialized tooling. You shouldn’t need a PhD and a custom jig to swap out a motor.

Designing for Decentralized Manufacturing

We want anyone to be able to reproduce Asimov from anywhere in the world. For under $25k USD.

Thus we used off the shelf components. And for the custom parts, to be feasible with affordable, low volume manufacturing processes. Most structural parts are compatible with plastic Multi Jet Fusion (MJF), a 3D printing process that produces strong, functional parts without custom tooling.

The knee plate, where stiffness and alignment are most critical, was redesigned several times. The initial design was a 2 part 3D design that required more complex tool paths, multiple setup changes, longer machine time, greater material removal, and additional assembly steps. We simplified the CNC machining requirement and turned the knee component into a plate instead.

The geometries are also amenable to alternatives like casting or traditional 3D printing, so teams without access to industrial MJF can still build the robot

Designing for Biorealism

Third, we treated humanaesthetics as a functional constraint. The main constraint was the largest motors size., i.e. if a robot is undersized its joints would look too knobbly.

From there, we worked backward to determine feasible link lengths, knee geometry, and motor placement. We used human joint range targets from anatomy and biomechanics references “Basis of Human Movement” to set joint limits and identify where forces should be carried through the structure.

• The smallest we could make the robot is 1.2 meters tall.

• The final robot (whole body) will have 26 degrees of freedom (DOF) and come in under 40kg.

• We anticipate low volume manufacturing to also come under $20,000 USD.

Leg Specifications & Cost

Each leg has 6 DOF to support human-like stance adjustment and foot placement. The knee is sized for approximately 40kg push capacity in demanding quasi-static cases like kneeling or hovering.

Mechanically, we packaged the leg actuators in a largely coplanar layout to simplify structure, load paths, and assembly.

The complete legs subsystem costs just over $10k USD all-in, roughly $8.5k for actuators and joint parts, the rest for batteries and control modules. We are not allowed to disclose Encos actuator pricing, and you will need to get a direct quote from the Encos team.

In total, our R&D over the last 100 days came under $30,000 USD. This spend also includes a hangar, tooling, and replacing parts. This part is amortizable for the whole body and other things. Making this super feasible for other scrappy labs like ours.

Ankles need RSU architecture

The ankles are the first to go. For humans and for robots.

Ankle design is critical because it directly determines how the foot transmits load and orientation to the ground during locomotion.

We chose a parallel RSU ankle (Revolute-Spherical-Universal) rather than a simple serial ankle with just one degree of freedom. RSU gives us two DOF ankles with both roll and pitch. Revolute actuators enable torque sharing between two motors working together to lift the whole body. This enables proximal placement of heavy components, which improves rigidity, precision, and dynamics. You also get better backdrivability characteristics, meaning the ankle can respond more naturally to ground contact forces.

Another good option would be have been linear actuators. With tradeoffs being that you’d get higher strength/stability, a more tendon sinewy look, but slower speed and a slightly more expensive component.

Toes make for an adaptive foot

We added a toe joint, and yes, this was very intentional.

Similar to how human toes passively bend to conform to the ground during stance, an articulated toe increases effective contact area and helps distribute load more evenly. But the real benefit comes during the stance-to-push-off transition.

As the body moves forward, a toe rocker helps the foot roll instead of pivoting on a rigid edge. This offloads peak forces, maintains contact, and increases effective contact area. The result is more stable traction and better forward propulsion.

We implement this as an extra toe hinge joint that is articulated but not actively actuated. It’s passive and compliant. Recent analyses of Tesla Optimus, for example, suggest an articulated toe box that behaves like a hinge with passive return (spring or elastic) rather than a fully motor-driven toe. You get the benefits of toe rollover without the complexity, mass, and control burden of adding another powered joint.

What we’re improving next

We are prioritizing hip and integration upgrades based on issues discovered in full-range testing for the next iterations.

The main change is at the hip. Our initial design mounted the hip-pitch actuator at roughly 45 degrees to better distribute upper-body load through the pelvis. This packaging constraint, however, limited the achievable hip flexion and prevented a good sitting posture. We’re moving to a horizontally mounted hip-pitch motor (about 90 degrees reoriented) to recover the required range of motion and improve usability for crouch and sit motions.

In parallel, we're tightening manufacturing and assembly quality. We missed some holes in early builds and need to improve jig and tolerance consistency. We're also upgrading the ankle pivot components from aluminum to steel for better wear resistance and strength margins.

Finally, we're shifting from leg-focused iteration to full-body integration. The complete assembly features 26 DOF across the whole body at under 40kg (exact mass still being finalized as we integrate components).

Want to go deeper?

We’d love your feedback

Join our Discord here to contribute to our open source efforts and give us feedback.

Share Asimov