Microduck-Inspired Sim-to-Real Biped Robot: Teach a Tiny Walking Robot New Tricks with Reinforcement Learning

At NVIDIA's GTC Berlin keynote on 21 October 2026, Jensen Huang asked four tiny robots to sit, stand and take a nap — and they obeyed. The robots were Microducks, built by Pollen Robotics (part of Hugging Face): a 25 cm, under-800 g open-source biped with 15 motors, a camera, an 8x8 time-of-flight sensor and two IMUs, sold as a $399 preorder, whose every movement is a policy trained in simulation and transferred to hardware. The point is not the duck shape; it is the method — sim-to-real learning — which a student build can honestly reproduce at servo scale. The problem: a $399 import with custom actuators is out of reach for most college budgets, and a hand-coded gait teaches nothing about how modern robots learn to move. This project closes that gap: a 3D-printed duck biped with MG90S servos, a Pi Zero 2 W, and a MuJoCo + PPO pipeline the student runs on a laptop, with the policy exported to the physical robot.

Microduck's reference numbers set the context: 25 cm tall, about 780 g, 15 motors, 7 shipped behaviors (walk, sit, stand, kick, pick-up, roller-skate, self-right), a 50 Hz onboard policy loop, and a training recipe Pollen documents openly — MuJoCo simulation, PPO, 4,096 parallel environments, roughly 1-2 hours on a single CUDA GPU, with domain randomization over battery sag, command delay and up to 1 degree of joint backlash (all reported). This build keeps the method and rescales the hardware honestly: 11 MG90S servos (5 degrees of freedom per leg plus a beak gripper) instead of 15 custom actuators, a Pi Camera and a single-point time-of-flight sensor as stand-ins for Microduck's camera and 8x8 lidar, and one MPU6050 IMU instead of two. What transfers is the real science — reward design, domain randomization, the sim-to-real gap, and measured transfer results — plus the experimental discipline the topic demands: tethered first transfers, fall trials, and a self-right success rate counted over real attempts, never claimed in advance.

A training run works like this. The student builds a MuJoCo model of the biped — same link lengths, masses and servo limits as the printed robot — and defines the reward: forward velocity, staying upright, and a penalty on jerky joint effort. PPO (Stable-Baselines3) trains the policy in simulation with domain randomization on — battery sag, joint backlash and command delay vary every episode — until the reward curve flattens. The trained policy is exported to ONNX and copied to the Pi Zero 2 W, which reads the MPU6050 and joint state, runs the policy at a 50 Hz loop (design target) and writes servo commands through the PCA9685 driver. Early transfers run under the tether; once the robot walks without it catching, the student runs the measured trials — walking distance before a fall, self-right success over 10 attempts, loop rate — and the dashboard logs every episode to CSV for the report's results section.

1. The student assembles the 3D-printed duck biped: 11 MG90S servos, Pi Zero 2 W, PCA9685 servo driver, MPU6050 IMU, Pi Camera and time-of-flight sensor, powered by a 2S LiPo through a UBEC.
2. A MuJoCo model matching the printed robot's link lengths, masses and servo limits is built, and the reward is defined: forward velocity plus upright posture minus joint-effort penalty.
3. PPO training runs on the laptop (Stable-Baselines3) with domain randomization — battery sag, joint backlash up to 1 degree and command delay — until the reward curve flattens.
4. The trained policy is exported to ONNX and deployed to the Pi Zero 2 W, which runs the observation-to-action loop at a 50 Hz loop (design target).
5. First transfers run under the overhead tether on a soft mat; the student compares simulated vs real gait and notes the reality gap honestly in the report.
6. Behaviors are switched from the dashboard or gamepad — walk, sit, stand, crouch/nap, kick, self-right — while joint telemetry and IMU data stream live.
7. Measured trials follow: walking distance before a fall, self-right success over repeated attempts and loop rate, all logged to CSV for the report.

Tech stack:

• 3D-printed duck biped body (PLA · 25 cm design target · beak gripper)
• 11x MG90S metal-gear servos (5 DOF per leg · beak gripper · 2.0 kg-cm at 6 V)
• Raspberry Pi Zero 2 W (quad-core Cortex-A53 at 1 GHz · 512 MB · Wi-Fi)
• PCA9685 16-channel PWM servo driver (I2C · 50 Hz servo pulses)
• MPU6050 6-axis IMU (posture sensing · fall detection · I2C)
• Pi Camera + VL53L0X time-of-flight sensor (camera and ranging stand-ins)
• MuJoCo simulation + PPO training (Stable-Baselines3 · ONNX export · domain randomization)
• Policy dashboard + gamepad control (behavior switching · telemetry · CSV episode log)

The biped's capabilities are six. First, sim-to-real walking: a PPO policy trained in MuJoCo walks the real robot, on the same observation-to-action loop Microduck uses, exported to ONNX and run at a 50 Hz loop (design target). Second, six switchable behaviors: walk, sit, stand, crouch/nap, kick and self-right, selected from the dashboard or a gamepad — the student-scale echo of the GTC voice commands. Third, domain randomization in training: battery sag, joint backlash and command delay are randomized so the policy survives the messiness of real servos. Fourth, live policy telemetry: joint angles, IMU pitch and roll, loop rate and battery voltage stream to the dashboard while the robot moves. Fifth, a fall-and-recover test rig: an overhead tether catches the robot during early transfers, and a dashboard fall button lets the student measure self-right success over repeated trials. Sixth, measurable transfer results: walking distance before a fall, self-right success rate and loop rate are logged per episode for the report.

• [Sim-to-real PPO walking] A PPO policy trained in a MuJoCo model of the biped walks the physical robot — reward-shaped for forward speed, upright posture and low joint effort, exported to ONNX and run at a 50 Hz loop (design target) on the Pi Zero 2 W.
• [Six switchable behaviors] Walk, sit, stand, crouch/nap, kick and self-right policies selectable from the dashboard or a USB gamepad — the student-scale stand-in for the sit/stand/nap voice commands demoed at GTC Berlin.
• [Domain randomization] Training randomizes battery sag, up to 1 degree of joint backlash and command delay (mirroring Microduck's reported recipe), so the transferred policy tolerates real servo slop instead of collapsing on first contact.
• [Live policy telemetry] Joint angles for both legs, IMU pitch/roll, policy loop rate and battery voltage stream to the dashboard in real time, with an episode event log for every policy switch, fall and self-right.
• [Tethered transfer rig] An overhead tether and soft floor mat catch the robot during the first sim-to-real transfers, so early policies can fail safely while the student measures the reality gap.
• [Fall-and-recover trials] A dashboard fall test plus automatic self-right policy lets the student count self-right successes over repeated trials — the project's headline measured result, reported honestly, never pre-claimed.
• [Beak gripper] A single-servo articulated beak picks up small light objects (design target: under 30 g), echoing Microduck's pick-up behavior at student scale.
• [Working policy dashboard] The dashboard (working demo UI included) switches policies, plots the training reward curve, shows live telemetry and logs every episode to CSV.

• Assembled duck biped: 3D-printed PLA body and head with beak gripper, 11 MG90S servos, Pi Zero 2 W, PCA9685 driver, MPU6050, Pi Camera, time-of-flight sensor, 2S LiPo with UBEC
• MuJoCo biped model matched to the printed robot plus the complete PPO training pipeline (reward definition, domain randomization, ONNX export)
• Policy dashboard app: behavior switching, live joint/IMU telemetry, training reward curve, fall test, CSV episode log (working demo UI included)
• Overhead tether transfer rig and soft mat procedure card with the staged transfer checklist
• Wiring diagram and biped assembly layout with servo calibration procedure
• Component list with Indian supplier sourcing notes for the servos, Pi Zero 2 W, driver board and sensors
• Project report PDF (sim-to-real method, reward design, domain randomization, measured transfer results, honest Microduck delta)
• PPT presentation for final review
• Viva Q&A preparation document (reinforcement learning, PPO, sim-to-real gap, domain randomization, why the policy fails on the real robot)