Why Uber’s self-driving car killed a pedestrian

By T.S.

THEY are one of the most talked-about topics in technology—but lately they have been for all the wrong reasons. A series of accidents involving self-driving cars has raised questions about the safety of these futuristic new vehicles, which are being tested on public roads in several American states. In March 2018 an experimental Uber vehicle, operating in autonomous mode, struck and killed a pedestrian in Tempe, Arizona—the first fatal accident of its kind. On May 24th America’s National Transportation Safety Board (NTSB) issued its preliminary report into the crash. What caused the accident, and what does it say about the safety of autonomous vehicles (AVs) more broadly?

The computer systems that drive cars consist of three modules. The first is the perception module, which takes information from the car’s sensors and identifies relevant objects nearby. The Uber car, a modified Volvo XC90, was equipped with cameras, radar and LIDAR (a variant of radar that uses invisible pulses of light). Cameras can spot features such as lane markings, road signs and traffic lights. Radar measures the velocity of nearby objects. LIDAR determines the shape of the car’s surroundings in fine detail, even in the dark. The readings from these sensors are combined to build a model of the world, and machine-learning systems then identify nearby cars, bicycles, pedestrians and so on. The second module is the prediction module, which forecasts how each of those objects will behave in the next few seconds. Will that car change lane? Will that pedestrian step into the road? Finally, the third module uses these predictions to determine how the vehicle should respond (the so-called “driving policy”): speed up, slow down, or steer left or right.

Of these three modules, the most difficult to build is the perception module, says Sebastian Thrun, a Stanford professor who used to lead Google’s autonomous-vehicle effort. The hardest things to identify, he says, are rarely-seen items such as debris on the road, or plastic bags blowing across a highway. In the early days of Google’s AV project, he recalls, “our perception module could not distinguish a plastic bag from a flying child.” According to the NTSB report, the Uber vehicle struggled to identify Elaine Herzberg as she wheeled her bicycle across a four-lane road. Although it was dark, the car’s radar and LIDAR detected her six seconds before the crash. But the perception system got confused: it classified her as an unknown object, then as a vehicle and finally as a bicycle, whose path it could not predict. Just 1.3 seconds before impact, the self-driving system realised that emergency braking was needed. But the car’s built-in emergency braking system had been disabled, to prevent conflict with the self-driving system; instead a human safety operator in the vehicle is expected to brake when needed. But the safety operator, who had been looking down at the self-driving system’s display screen, failed to brake in time. Ms Herzberg was hit by the vehicle and subsequently died of her injuries.

The cause of the accident therefore has many elements, but is ultimately a system-design failure. When its perception module gets confused, an AV should slow down. But unexpected braking can cause problems of its own: confused AVs have in the past been rear-ended (by human drivers) after slowing suddenly. Hence the delegation of responsibility for braking to human safety drivers, who are there to catch the system when an accident seems imminent. In theory adding a safety driver to supervise an imperfect system ensures that the system is safe overall. But that only works if they are paying attention to the road at all times. Uber is now revisiting its procedures and has suspended all testing of its AVs; it is unclear when, or even if, it will be allowed to resume testing. Other AV-makers, having analysed video from the Tempe accident, say their systems would have braked to avoid a collision. In the long term, AVs promise to be much safer than ordinary cars, given that 94% of accidents are caused by driver error. But right now the onus is on Uber and AV-makers to reassure the public that they are doing everything they can to avoid accidents on the road to a safer future.

Dig deeper
What it’s like to ride in a self-driving Uber (Mar 2018)
Why self-driving cars will be mostly shared, not owned (Mar 2018)
Reinventing wheels: a special report on self-driving cars (Mar 2018)