In this guide
How Autonomous Racing Drones See: Single-Camera Vision, Optical Flow and the MonoRace Breakthrough
On October 14, 2026, TU Delft's Micro Air Vehicle Lab published a result that reset expectations for robot autonomy: MonoRace, a racing drone that beat three former human world-champion pilots using nothing but a single forward-facing camera and an inertial measurement unit. No GPS. No external motion tracking. Just one camera, one motion sensor, and a neural network commanding the motors 500 times per second. This guide explains how that is possible, why drone racing is the hardest test in autonomy, and how a student can build an honest lab-scale version of the same ideas.
Why racing is the hardest autonomy problem
Most autonomous robots move slowly enough that perception can lag. A warehouse robot crawling at 1 m/s has hundreds of milliseconds to decide what to do next. A racing drone at 100 km/h covers nearly 28 metres every second — at that pace, a perception delay of even a few tens of milliseconds means the difference between threading a gate and clipping it. Racing compresses every autonomy problem — sensing, estimation, planning, control — into the tightest possible time budget, which is exactly why researchers use it as a benchmark. Any system that flies a racecourse reliably has solved problems that transfer directly to self-driving cars, warehouse drones and mobile robots. MonoRace matters not because drone racing needs better drones, but because it proves how much performance can be squeezed out of deliberately minimal sensing and computing.
What MonoRace actually did
MonoRace is the system built by TU Delft PhD researchers (main author Stavros Bahnam, published in Science Robotics, October 14, 2026). The headline facts, cross-checked across multiple outlets:
- Sensor suite: one forward-facing camera plus an IMU (accelerometer + gyroscope). No stereo vision, no GPS, no external tracking cameras.
- Control: a neural network that learns to command the motors directly — there is no separate "plan a path, then follow it" stage. The network sees pixels and IMU readings and outputs motor commands.
- Speed: motor commands issued about 500 times per second; top speeds around 100 km/h on the course.
- The result: at the 2025 Abu Dhabi Autonomous Drone Racing Competition, MonoRace beat every rival autonomous team and posted faster times than three former human world-champion pilots — the first AI-piloted drone to beat human pilots in an international racing competition.
- Hardware honesty: the competition drone ran 5.1-inch propellers with a Jetson Orin NX companion computer handling vision and a neural controller on the flight controller itself.
The deeper point the Delft team made: smart robots may not need big, power-hungry computers. MonoRace's achievement is as much about efficiency as about speed — compact AI doing high-performance control in messy real-world conditions.
How single-camera navigation works (the core idea)
A camera alone cannot tell you where you are — it gives you a flat image, not distances. MonoRace-style systems recover position in two complementary ways:
1. Gate detection from the image. A neural network scans each frame and finds the racing gates — their position and apparent size in the image. A gate that appears large and centred is close and straight ahead; one that is small and off to the side is far away or off-course. This is the same information a human FPV pilot reads instinctively from the video feed.
2. Motion estimation from the IMU. The inertial measurement unit feels acceleration and rotation hundreds of times per second. By integrating these readings, the drone estimates how it is moving between camera frames. IMU data drifts over time (small errors accumulate), but fused with the visual gate fixes, the drift is constantly corrected — each gate sighting re-anchors the estimate.
The fusion of "where are the gates in my image" with "how am I moving" is what lets the drone fly without GPS. Outdoors, GPS gives absolute position; indoors or at racing speed, GPS is too slow and too coarse anyway. MonoRace's neural network learned this entire mapping — pixels plus IMU in, motor commands out — end to end.
Why students don't train the neural version (and what they build instead)
Training an end-to-end racing controller like MonoRace's requires a flight simulator, enormous amounts of simulated racing data, serious GPU compute, and expertise in reinforcement learning. A student lab has none of these at the required scale — and when a learned controller fails, there is nothing inspectable to debug. That is why the honest student version, like our MonoRace-inspired autonomous racing drone project, uses a classical pipeline that mirrors MonoRace's architecture with debuggable parts:
| MonoRace (research) | Student version (classical) |
|---|---|
| Learned gate detector (neural net) | ArUco fiducial markers + OpenCV detection |
| Learned pose estimation | PMW3901 optical-flow sensor + IMU fusion |
| Neural direct motor control at 500 Hz | PID velocity control at ~30 Hz |
| 100 km/h, 5.1-inch racer | 4–6 m/s cap, prop-guarded indoor cinewhoop |
Every row is a deliberate, documented downgrade — and each one is exactly what makes the student build defensible in a viva: every stage produces numbers the student can log, plot and explain.
How ArUco markers give a drone its position
An ArUco marker is a square black-and-white pattern with a unique ID, generated by OpenCV's ArUco module. Because the pattern's real-world size is known (you print it at, say, 15 cm square), a single camera image is enough to recover the marker's full 3D pose relative to the camera:
- Bearing comes from where the marker appears in the image (left/right, up/down).
- Range comes from how large it appears — a 15 cm marker filling half the frame is much closer than one occupying a tenth of it.
- Orientation comes from the marker's apparent perspective distortion.
Stick one marker on each racing gate and the drone gets a fresh range-and-bearing fix for the next gate on every frame it is visible — about 24 times per second on a Raspberry Pi Zero 2 W in the reference build. The honest limitation: the drone cannot navigate gates it cannot see, and plain gates without markers are invisible to it. MonoRace's neural detector has no such crutch — which is precisely why the ArUco approach is listed as the student-scale stand-in, not the equivalent.
How optical-flow sensors hold a drone still
GPS-denied hover is the second half of the problem. The PMW3901 (PixArt) is a tiny optical-flow sensor — essentially the same principle as a computer mouse sensor. It continuously captures low-resolution images of the floor and computes how many pixels the surface texture shifted between frames, reporting X/Y motion deltas over SPI. Fused with the IMU (which gives tilt and acceleration), the flight controller can hold position over a point with no GPS at all.
Two physical facts every student builder must know:
- It needs texture. Optical flow measures texture motion. A plain white floor has no features to track, so hover drifts. The standard fix is a printed high-contrast mat under the course — in practice the single biggest stability improvement in the bench build.
- It measures angle, not distance. The same pixel shift means different real-world motion at different heights, so the sensor needs an altitude estimate (from a rangefinder or the IMU/barometer fusion) to convert pixel deltas into metres per second. Its useful working range starts around 80 mm above the surface.
PID velocity control in plain words
With gate fixes arriving from the camera and hover stability from optical flow, the last piece is steering. A PID controller is the workhorse here, and each letter does one intuitive job for a drone chasing a gate:
- P (proportional): steer harder the further off-course you are. Big bearing error to the gate, big correction.
- I (integral): fix persistent small errors — the slow drift that P alone never quite removes, like a constant side wind from the drone's own prop wash near walls.
- D (derivative): dampen the correction as you approach the target, so the drone doesn't overshoot the gate centre and oscillate.
Tuning these three gains is the core lab work of the project: too much P and the drone oscillates through gates; too much D and it reacts sluggishly; the tuning log itself becomes original data for the report. The commands leave the companion computer as velocity setpoints over a serial/MAVLink offboard link to the flight controller, which handles the fast inner stabilization loop.
The human-vs-AI race format: how you evaluate it
MonoRace's cultural impact came from the format, not just the engineering: AI against human champions, same course, timed laps. The student version copies the evaluation method because it is genuinely the right way to measure an autonomy system:
- Fly the 6-gate course in autonomous mode; log every lap time and gate-pass timestamp.
- Fly the identical course in manual FPV mode with the vision loop disengaged; log the same data.
- Compare best laps side by side on the dashboard.
The honest methodology notes: run both modes the same day on the same course, because lighting changes move the vision detector's performance and battery voltage moves lap times. A comparison across different days or courses proves nothing — stating the procedure is part of the engineering.
Building the honest version
The complete lab-scale build — ducted cinewhoop airframe, Pi Zero 2 W vision stack, PMW3901 optical flow, 6-gate ArUco course kit, race-control dashboard with the human-vs-AI mode, and the tuning procedure — is documented as MonoRace-Inspired Autonomous Racing Drone: Self-Flying Quadcopter with Vision-Based Gate Navigation, including the two builder notes from the bench build (the floor-mat fix and the 470 µF capacitor on the 5 V rail against brownout resets). If you are writing the project report, the sections above map directly onto it: MonoRace as the literature reference, the classical pipeline as your methodology, the tuning log as your results, and the human-vs-AI table as your evaluation.
Frequently asked questions
Do I need machine learning for an autonomous racing drone?
No — and that is the point of the classical pipeline. ArUco detection, optical-flow fusion and PID control contain no trained models at all, yet they produce a genuinely autonomous gate-racing drone. Machine learning enters as documented future scope: train a gate detector in simulation and keep the classical pipeline as the safety fallback.
Why not just use GPS?
GPS is unavailable indoors, updates at only a few hertz, and its metre-scale accuracy is useless for threading gates at speed. Racing autonomy is GPS-denied by nature — which is why optical flow plus vision is the real solution, not a workaround.
What is the minimum viable sensor set?
One forward camera plus the flight controller's built-in IMU is the absolute minimum (the MonoRace configuration). The optical-flow sensor is what makes indoor hover practical for a student build; without it, position hold between gates depends entirely on the IMU, which drifts within seconds.
How fast can the student version actually fly?
The reference build caps speed at 4–6 m/s (a design target, not a guarantee) for a netted indoor course. That is roughly 15–20 km/h — far from MonoRace's 100 km/h, and deliberately so. Indoor safety hardware (prop guards, nets, kill switch) sets the ceiling, not the software.
Can this approach work outdoors?
The vision and control pipeline transfers, but outdoor autonomous flight raises regulatory and safety questions a student build should not claim to solve. The honest scope is a netted indoor course; outdoor line-of-sight FPV with the same airframe is a separate, manually-flown activity.
Limitations
- This guide describes the classical student pipeline, not MonoRace's end-to-end neural controller — the two differ in speed, agility and robustness, and the guide states the delta explicitly rather than implying equivalence.
- Optical-flow hover needs a textured floor and adequate light; the sensor cannot see in the dark and drifts over featureless surfaces.
- ArUco-based navigation only works where marked gates are visible; it does not generalize to unmarked environments the way a learned detector does.
- All speed, rate and range figures in the student build are design targets from a bench build, not guaranteed specifications — lighting, gate spacing and PID tuning move every number.
- Nothing here authorizes outdoor autonomous flight; drone regulations apply and the build is scoped to controlled indoor lab use.