The Self-Driving Car Story Started Earlier Than You Think

Decades before Waymo and Tesla, a Mercedes van used cameras and computers to drive itself down the autobahn.

Photo by: Ralph Hermens

By: Andrei Nedelea

Jun 4, at 11:26am ET

Add InsideEVs as a preferred source in Google

Comment

Tesla may be widely associated with its very public quest to automate driving, but the origin of self-driving cars goes back to before the Silicon Valley startup even existed. A Mercedes drove autonomously on public highways for 1,000 miles between Germany and Denmark—sometimes at autobahn speeds—back in 1995.

Before that, in 1987, Ernst Dickmanns’ VaMoRs research vehicle drove itself down an unopened stretch of autobahn using cameras, onboard computers, and automated control of the steering, throttle, and brakes. It reached 60 mph, covered more than 12 miles, and did so with less computing power than your smartwatch has.

The first true experimental semi-autonomous car is even older, though. In 1977, the Tsukuba Mechanical Engineering Laboratory in Japan made an early vision-guided autonomous test vehicle. It used images from two onboard cameras and followed guide features on the road. It was primitive and slow, but it drove autonomously (albeit in a controlled, structured environment) using on-board tech.

We try not to throw around the terms “self-driving” and “autonomous” too liberally here, simply because most consumer-facing systems are not actually legally capable of driving themselves. These vehicles still exist on a spectrum somewhere between basic driver assistance and genuinely driverless autonomy, with most of today’s systems still much closer to helping a human drive than replacing one entirely.

Today’s robotaxis didn’t appear out of the blue, and it’s not just Tesla that deserves credit for hastening its arrival. It’s a collection of experiments carried out over the years, many of which were way ahead of their time, each building toward an end goal: Making a car that can see its surroundings, understand what it sees, and adapt to a constantly changing environment so it can drive itself.

Japan Got There First, With One Big Catch

What researchers from the Tsukuba Mechanical Engineering Laboratory built in 1977 is definitely a milestone, especially given the time when it was built. This experimental vehicle had two roof-mounted cameras to “see” road markings, and the information was processed by an onboard computer.

The catch was that it had a top speed of 30 km/h (under 20 mph) and operated only on a closed test track. But unlike previous attempts, it had no wires and wasn’t remotely operated. Calling it autonomous is a stretch, though, because all it could do is follow road markings and couldn’t adapt to other vehicles or changes in its environment. But it did set the stage for more advanced experiments that followed.

The Japanese researchers never followed it up with a new design, though, and the self-driving car story moved to Europe in the 1980s.

The Mercedes Van That Changed The Road Ahead

A further breakthrough occurred in Germany. VaMoRs, short for Versuchsfahrzeug für autonome Mobilität und Rechnersehen—”Test Vehicle for Autonomous Mobility and Computer Vision” in English—was another academia-led project started by Ernst Dickmanns at the University of the Bundeswehr Munich.

Dickmanns and his team equipped a Mercedes-Benz van with cameras and sensors, as well as systems to allow it to control its steering, braking, and speed. In 1986, this experimental van drove itself for the first time, and one year later, it hit 60 mph (the van's maximum speed) on its own while driving on a closed stretch of the autobahn for over 12 miles (20 km).

Over time, it gained additional autonomous functions, such as detecting obstacles ahead, adjusting its speed to match the vehicle in front, and performing automatic lane changes.

Europe Tried To Build The Future Before Silicon Valley

Dickmanns and his team then moved on to contribute to the Eureka PROMETHEUS project, a much larger operation that involved over 600 companies, including several major automakers, one of which was Mercedes-Benz. This project picked up where VaMoRs left off, but it abandoned the van in favor of two S-Class-based vehicle testbeds.

They used cameras and onboard computers to read the road, track surrounding traffic, and control the car on highways. For the early 1990s, this was wildly advanced: the cars constantly monitored vehicles ahead and behind at up to 330 feet (100 meters), followed lanes, and evaluated whether a lane change was safe.

In 1994, the two test vehicles drove autonomously on busy three-lane highways around Paris (with safety drivers), doing the speed limit of 80 mph (130 km/h), keeping up with traffic, and changing lanes autonomously. In his paper “Vehicles Capable of Dynamic Vision,” Dickmanns notes that the vehicle showed “remarkable performance” in regular highway traffic. By 1994, they could recognize road curvature, lane width, the number of lanes, lane markings, their own position relative to the lane and road, and the position and speed of up to 10 other vehicles—five ahead and five behind.

The 1995 trip from Munich, Germany, to Odense, Denmark, and back was the project’s biggest achievement. Around 95% of the 1,000 miles (1,600 km) between the two cities was driven autonomously, and the vehicles even reached 112 mph (180 km/h).

Dickmanns explains in the paper that, despite its advanced state, the system still had limitations. Because the system used only black-and-white video and edge feature extraction (basically just looking for lines and borders), it could not handle construction sites where yellow temporary markings were overlaid on white ones. It also had trouble with vehicles cutting into its lane very close ahead, partly because they could not always be detected early enough and partly because its range estimate needed time to adjust.

This is where the European chapter of early autonomous driving starts to wind down. But Mercedes’ involvement in PROMETHEUS was clearly put to good use. A few years later, the company introduced Distronic, the world’s first adaptive cruise control system in a production vehicle, which debuted on the 1998 W220 S-Class. This system allowed the vehicle to maintain a safe distance from the car in front. However, since Distronic was radar-based rather than using cameras, its link to PROMETHEUS is more tangential. The approach may have differed, but it still represented the same broader goal of automating aspects of driving.

DARPA Made Self-Driving Cars An American Obsession

In the United States, the story of autonomous driving took a similar route, initially starting with universities before moving into defense research and eventually reaching Silicon Valley.

Instead of a huge carmaker-led European project, America’s self-driving push was shaped by university robotics labs, military-backed competitions, and, eventually, tech companies that saw autonomy less as a driver-assistance feature and more as a software problem waiting to be solved.

Carnegie Mellon University was the key early player. Its Navlab program began in the 1980s and culminated in the 1995 “No Hands Across America” run, in which a modified Pontiac Trans Sport drove from Pittsburgh to San Diego, with the computer handling most of the steering while humans still controlled the throttle and brakes.

The closest thing the U.S. had to the PROMETHEUS project in Europe was the National Automated Highway System. In 1991, Congress approved $650 million to develop the technology needed for fully autonomous cars. Having self-driving cars was thought to not only improve traffic safety but also fuel efficiency.

In 1997, its Demo ’97 event used a 7.6-mile HOV lane section of I-15 in San Diego, allowing a convoy of automated test vehicles to run in a controlled highway environment. A Transportation Research Board report describes the event as a “Proof of Feasibility Demonstration,” adding that thousands of magnets were added to the road to make it possible.

It was all quite rudimentary and didn’t really move the autonomous driving needle. The Smithsonian Magazine explains that the program never delivered automated highways to Americans, partly because the legislation required a 1997 demonstration without clearly defining what a “fully automated highway system” actually meant.

It was the Defense Advanced Research Projects Agency (DARPA) that really kick-started the American story, though. Its first unmanned ground vehicle, the 1985 Autonomous Land Vehicle (ALV), was a military-minded project aimed at creating a vehicle that could navigate outdoors without a driver or teleoperation.

DARPA's ALV

Photo by: Lockheed Martin

In 2004, DARPA organized the Grand Challenge, in which autonomous vehicles navigated a 142-mile course from just outside Barstow, California, to Primm, Nevada. None of the 15 vehicles entered in the competition even came close to finishing in the required 10 hours. The best performer was Carnegie Mellon’s Sandstorm, which made it only about 7.4 miles before getting stuck.

The first edition was a flop, but the 2005 Grand Challenge saw five vehicles complete a 132-mile desert course. Stanford University’s Stanley vehicle, basically a sensor-laden Volkswagen Touareg, finished the course in just under 7 hours. DARPA frames this as the breakthrough that proved autonomous ground vehicles could complete a long, difficult off-road route without human control.

But while driving through the desert autonomously was impressive, it did little to advance self-driving road cars. DARPA then launched the 2007 Urban Challenge, in which autonomous vehicles had to navigate a mock urban environment. The winning vehicle was a modified Chevy Tahoe from Carnegie Mellon University, which completed the course in 4 hours and 10 minutes at an average speed of 14 mph.

The vehicles taking part in the 2007 Urban Challenge were already starting to look a lot like today's robotaxi vehicles, most of which are still heavily modified passenger vehicles. They had the same massive array of sensors and cameras on the roof, as well as radars and sensors lower down on the vehicle. They didn’t look all that different from the Jaguar i-Pace that Waymo still uses today.

Silicon Valley Turned Autonomy Into A Business

Having read all of the above, you now know that Silicon Valley didn’t invent autonomous vehicles. But it did make a business case for them, giving an incentive for investment. By 2009, Google had already launched its self-driving car project, which was described at the time as a “moonshot” experiment aimed at revolutionizing safety.

It used a combination of lidar, radar, cameras, and GPS to navigate and was essentially born from technologies developed during the DARPA challenges of the early 2000s. But while Google initially used a bespoke, tiny two-door car with no steering wheel, the project shifted to using existing vehicles modified for autonomous driving.

When Google’s self-driving project became Waymo in late 2016, it was not alone. Uber had also begun offering (invite-only) autonomous rides in Pittsburgh a few months prior, using a fleet of modified Ford Fusion sedans that had engineers monitoring the drive from the front seats.

In October 2016, Tesla announced that all cars it would build after that point would have the hardware needed for what it described as “full self-driving.” However, it took another four years for Full Self Driving Beta to roll out, initially to a small group of selected U.S. users. Even then, it hardly lived up to the name. A decade after that first announcement, Tesla customers using FSD are still legally driving, and therefore still responsible for the car’s actions.

The Hard Part Still Isn’t Over

The self-driving car didn’t suddenly arrive when Google put spinning lidar units on Priuses, or when Tesla started selling the promise of Full Self-Driving. From Tsukuba’s slow, rudimentary, camera-guided test car to Dickmanns’ Mercedes van, from PROMETHEUS to Navlab, from DARPA’s desert failures to Waymo’s robotaxis, the same basic problem was tackled from different angles.

What do you think?

View

Comments

Even today, when vehicles are closer than they’ve ever been to actually driving themselves, we’re still wary of calling them truly autonomous. They’re very good, but the ones you can buy aren’t legally self-driving, and the truly autonomous ones are limited to a select number of cities.

In another five years, we will be much closer to having fully autonomous vehicles, but will they be truly fully autonomous? There’s also a good chance that it may not come from America. It could be Chinese.