Tesla Autopilot v8.0 Released: Adds Advanced Radar Over Camera Reliance, Stronger Warnings

SEP 11 2016 BY JAY COLE 67

Tesla announced its long awaited Autopilot v8.0 update today  – a system that will now rely significantly more on radar (rather than cameras) that its predecessor (full Tesla press release/statement below).

The radar sensors required for this upgrade are, for the most part, already installed on Tesla Model S and Model X Autopilot-enabled vehicles; at least all those with a manufacture date of October 2014 or later with the functionality.  Up until now the sensors included in the Autopilot package played 2nd fiddle to the on board cameras.

Autopilot In Trickier Locations Like The Aberdeen Tunnel Will Now Gain Knowledge From The Entire Tesla Fleet's Historical Use Of The Location

Autopilot In Trickier Locations Like The Aberdeen Tunnel Will Now Gain Knowledge From The Entire Tesla Fleet’s Historical Use Of The Location

With the Autopilot update, radar takes over the main controls, which will better enable the car to travel in bad weather conditions than could be achieved with just a camera according to the company.

From the Q&A Tesla CEO Elon Musk said of the feature:

“Even if you’re driving down the road and the visibility is very low and there’s a multi-car pile up, the camera can’t see it, but the radar would and apply the brakes.”

Previously the company relied less on radar due to frequency of false positives being detected, something that would happen far less in earlier systems using a camera-based system.

The new radar system has a more detailed point cloud, which Tesla says unlocks access to six times as many radar objects, with a lot more information per object as well.  Additionally the system will now take a radar snapshot every tenth of a second to determine objects, and whether they are standing still or in motion, as well as to exclude more “spurious reflections”.

The Tesla fleet of cars will also now build a database non-threatening objects and landmarks: roads signs, bridge underpasses and other obstacles that might appear to Autopilot as having the potential for a collision event, but that received no response via its live human driver when encountered, in order to respond more appropriately in the future while activated.

“Initially, the vehicle fleet will take no action except to note the position of road signs, bridges and other stationary objects, mapping the world according to radar. The car computer will then silently compare when it would have braked to the driver action and upload that to the Tesla database. If several cars drive safely past a given radar object, whether Autopilot is turned on or off, then that object is added to the geocoded whitelist.”

Autopilot 8 will now be able to achieve "car ahead", 2 car tracking

Updated Autopilot functionality will now be able to achieve “car ahead”, 2 car tracking

Tesla has also stepped up the punitive action for not following the terms and conditions to operate Autopilot after some high profile accidents (sometimes fatal) have occurred when the driver was not properly governing the EV – something Elon Musk said on the call happens more frequently with “veteran” users.

With the Autopilot version 8 release, the auto-steering system will now disengage if the driver of the car ignores several warnings (3 of them over an hour) to keep contact with the steering wheel.  And to re-engage the software, the car must first be pulled off the road and put into park to be reset.

“We’re making much more effective use of radar. I am highly confident this will be a substantial improvement.” – Musk said on the conference called, while adding, “It is quite unequivocal that Autopilot improves safety, and with this update, it improves it even more.”

During the call, the Tesla CEO was asked if the recent fatal crash, with Autopilot enabled, that involved Josh Brown’s Model S and a semi-truck would have been prevented with this new radar system and new warning system, Musk stated that “we believe it would have”.

As always, Musk was careful to state that Autopilot is not a guarantee of 100% safe transportation, 100% of the time.

“I do want to emphasize that this does not mean perfect safety.  Perfect safety is really an impossible goal. It is about improving the probability of safety.   That’s really all you can accomplish.”

No hard timeline was given for Autopilot 8’s roll-out, other than “in the next week or two”.


Full Tesla Press Release on AutoPilot v8.0 

Upgrading Autopilot: Seeing the World in Radar

Autopilot 8.0 arrives shortly this September

Autopilot 8.0 arrives shortly this September

While there are dozens of small refinements with Version 8 of our software, described in addendum below, the most significant upgrade to Autopilot will be the use of more advanced signal processing to create a picture of the world using the onboard radar. The radar was added to all Tesla vehicles in October 2014 as part of the Autopilot hardware suite, but was only meant to be a supplementary sensor to the primary camera and image processing system.

After careful consideration, we now believe it can be used as a primary control sensor without requiring the camera to confirm visual image recognition. This is a non-trivial and counter-intuitive problem, because of how strange the world looks in radar. Photons of that wavelength travel easily through fog, dust, rain and snow, but anything metallic looks like a mirror. The radar can see people, but they appear partially translucent. Something made of wood or painted plastic, though opaque to a person, is almost as transparent as glass to radar.

On the other hand, any metal surface with a dish shape is not only reflective, but also amplifies the reflected signal to many times its actual size. A discarded soda can on the road, with its concave bottom facing towards you can appear to be a large and dangerous obstacle, but you would definitely not want to slam on the brakes to avoid it.

Tesla Model S w/ Autopilot

Tesla Model S w/ Autopilot

Therefore, the big problem in using radar to stop the car is avoiding false alarms. Slamming on the brakes is critical if you are about to hit something large and solid, but not if you are merely about to run over a soda can. Having lots of unnecessary braking events would at best be very annoying and at worst cause injury.

The first part of solving that problem is having a more detailed point cloud. Software 8.0 unlocks access to six times as many radar objects with the same hardware with a lot more information per object.

The second part consists of assembling those radar snapshots, which take place every tenth of a second, into a 3D “picture” of the world. It is hard to tell from a single frame whether an object is moving or stationary or to distinguish spurious reflections. By comparing several contiguous frames against vehicle velocity and expected path, the car can tell if something is real and assess the probability of collision.

The third part is a lot more difficult. When the car is approaching an overhead highway road sign positioned on a rise in the road or a bridge where the road dips underneath, this often looks like a collision course. The navigation data and height accuracy of the GPS are not enough to know whether the car will pass under the object or not. By the time the car is close and the road pitch changes, it is too late to brake.

This is where fleet learning comes in handy. Initially, the vehicle fleet will take no action except to note the position of road signs, bridges and other stationary objects, mapping the world according to radar. The car computer will then silently compare when it would have braked to the driver action and upload that to the Tesla database. If several cars drive safely past a given radar object, whether Autopilot is turned on or off, then that object is added to the geocoded whitelist.

When the data shows that false braking events would be rare, the car will begin mild braking using radar, even if the camera doesn’t notice the object ahead. As the system confidence level rises, the braking force will gradually increase to full strength when it is approximately 99.99% certain of a collision. This may not always prevent a collision entirely, but the impact speed will be dramatically reduced to the point where there are unlikely to be serious injuries to the vehicle occupants.

The net effect of this, combined with the fact that radar sees through most visual obscuration, is that the car should almost always hit the brakes correctly even if a UFO were to land on the freeway in zero visibility conditions.

Taking this one step further, a Tesla will also be able to bounce the radar signal under a vehicle in front – using the radar pulse signature and photon time of flight to distinguish the signal – and still brake even when trailing a car that is opaque to both vision and radar. The car in front might hit the UFO in dense fog, but the Tesla will not.

 


 

Additional Autopilot Release Notes:

  • TACC braking max ramp rate increased and latency reduced by a factor of five
  • Now controls for two cars ahead using radar echo, improving cut-out response and reaction time to otherwise-invisible heavy braking events
  • Will take highway exit if indicator on (8.0) or if nav system active (8.1). Available in the United States initially
  • Car offsets in lane when overtaking a slower vehicle driving close to its lane edge
  • Interface alerts are much more prominent, including flashing white border on instrument panel
  • Improved cut-in detection using blinker on vehicle ahead
  • Reduced likelihood of overtaking in right lane in Europe
  • Improved auto lane change availability
  • Car will not allow reengagement of Autosteer until parked if user ignores repeated warnings
  • Automatic braking will now amplify user braking in emergencies
  • In manual mode, alerts driver if about to leave the road and no torque on steering wheel has been detected since Autosteer was deactivated
  • With further data gathering, car will activate Autosteer to avoid collision when probability ~100%
  • Curve speed adaptation now uses fleet-learned roadway curvature
  • Approximately 200 small enhancements that aren’t worth a bullet point

Video Bonus: Chris/KmanAuto takes the time to breakdown the entire Tesla release notes/blog and explain it all in simpler terms

Categories: Tesla

Tags: ,

Leave a Reply

67 Comments on "Tesla Autopilot v8.0 Released: Adds Advanced Radar Over Camera Reliance, Stronger Warnings"

newest oldest most voted

An interesting approach using brute force to basically map everything. Well that’s computers are really good at.

Anybody give pause to reflect on just how much it will cost to repair the autopilot if it breaks? Nice idea but at what cost for autopilot cameras, radar, etc.

This system is so far ahead of any other.

I’m glad he’s not shutting it down too abruptly. 3 “hold wheel” warnings within an hour is already pretty rare, as far as I’m concerned.

Holding the wheel and paying attention are two different things. In almost 100% of accidents due to inattentiveness, the driver’s hands were on the wheel.

That potentially changes with Autopilot, but in the one fatality when autopilot was in use, it wasn’t the steering that was the issue. The car kept itself on course and holding the wheel wouldn’t have made a bit of difference.

It would be nice if the car had a way of assuring that drivers were paying attention, but the idea of checking to see if a driver’s hands are on the wheel is more for show than for safety. If anything, cars with no automated features would be the ones where you’d most want to assure that the driver’s hands are on the wheel.

A driver who is watching the road will instinctively grab the wheel if autopilot isn’t doing what’s expected. A driver who isn’t watching the road could be holding the wheel but it won’t make a difference.

Suggestion – “Drowsy Driver Warning System” – https://youtu.be/wfVAXxaS7BE – Uploaded on 7 Nov 2008 (Upgraded to Tesla’s best)!

Ultimately, if “Autopilot” requires you to hold the wheel and pay attention while you watch the road, it’s literally inferior to normal driving.

At that point, it’s the same as normal driving, only with the added “bonus” of the car randomly choosing to do something unexpected that you have to react to. It’s like a sick version of a video game.

You’d think so. But you’d be wrong.

Spider-Dan
September 12, 2016 at 7:44 pm

Ultimately, if “Autopilot” requires you to hold the wheel and pay attention while you watch the road, it’s literally inferior to normal driving.

Well, I think I’m going to go with what any number of automobile reporters and Tesla owners have reported… rather than what one random anonymous person on the Internet says; someone who I am guessing has no experience driving a car with Tesla Autopilot/AutoSteer.

(Not that I’ve driven one, either; just ridden in one. But then, I’m not the one who is contradicting all those who have driven one and say it’s a definite improvement.)

I am certainly open to correction.

In what way is an “autonomous” system that requires you to hold the wheel and watch the road – a system with some rather high profile failures with severe consequences – better than holding the wheel and watching the road as you drive yourself?

The only “improvement” I can see is if you are using the software incorrectly; i.e. not paying attention.

Speaking from experience, it’s better because you don’t have to constantly make tiny little muscle movements to keep the car on the road, as you do when you’re driving manually. Over the course of hours, these add up to a lot of stress. It’s analogous to sitting in a chair instead of standing up straight – in either case, you have to use some muscles to keep yourself from falling over, but not very many people would tell you sitting for hours is just as fatiguing as standing. (Oh please, let’s not get into a discussion about the alleged health risks of sitting! It’s an analogy, if you don’t like it pick another one.) Empirically, I feel less fatigued after ten hours on the highway with Autopilot than I used to after four hours without. Others have speculated that the level and quality of attention you have to provide the road is also different – the car is the worker and you are the supervisor, instead of being the worker yourself. This sounds very plausible to me, and may play into what I experience as reduced fatigue, but I couldn’t prove it one way or another. Some people have gone… Read more »

One minor point is that this hasn’t been released. It was announced today and may be released within the next one to two weeks.

So now Tesla has UFO avoidance?

Alien dreadnougts, billions of hamsters, UFOs? We’re doomed man, DOOMED!?

Is this the Autopilot 2.0 that was said to come out this summer? Cause I thought that involved new hardware as well.

No, this is just an upgrade of the software to Autopilot 1.0. Were have you heard that autopilot 2.0 would come this summer? Last year Elon said that he belived autopilot would be ready with in two years, so sometimes in the end of next year. Tesla will probebly start add the hardware on new cars earlier but we don’t know when. It have been mush speculation that autopilot will be announced tougher with the second announcement of Model lll but that’s not for sure and it doesn’t mean that it will be ready for customers then.

Can’t remember, but I thought I read that Elon will introduce a major upgrade to autopilot this summer and that it would include upgrade hardware. I don’t think he used the term autopilot 2.0 but others did. I’m guessing I miss read or miss remembered. I kinda wish there is a major update to the hardware but I guess it makes sense not to update hardware when you can still get much more out of the hardware. Hopefully the new hardware will be out before the model 3, would kinda suck if you were first in line for a model 3 and you find out 6 months later they updated the hardware on autopilot.

He said that AP 2.0 will be announced toward the end of the year.

The Hardware update is coming. People on Tesla Motors club have already seen new forward facing camera mounts that hold 3 cameras.

I am guessing it will go into production on S and X before the announcement, and also be available on 3 at release. So roughly on S and X for 1 full year before 3, tempting people to jump at S or X.

From what I understand 2014 was the first year that had autopilot hardware installed. So according to this article October 2014 is the earliest hardware that can take the rev 8 software. I guess that means there is roughly 6 months or so of cars that can’t be updated.

Anyone concur? This is kind of important if one is shopping for a used Tesla that has auto pilot.

Tesla don’t have model year like other manufacturers, tesla introduce new features in the cars when they like and they started to install autopilot hardware in October 2014 so this is for all cars that have autopilot and there is a lot of cars that was produced in 2014 that don’t have autopilot.

@Viktor
OK thx

“With further data gathering, car will activate Autosteer to avoid collision when probability ~100%”

This is huge, and will significantly improve the safety of Autopilot enabled cars. I’m quite excited. Autopilot finally has a shot at producing fewer fatalities per mile than non-Autopilot.

Notice how this serial Tesla basher not so subtly implies that you’re safer not using Tesla Autopilot than using it. Of course, that’s just fraudulent FUD, and isn’t based in fact at all.

Great news, but it underscores how Autopilot was rushed to market. Autopilot had a known defect, which Mobileye warned Tesla about. In fact, Mobileye was so upset that Tesla disregarded their warnings that they terminated their relationship with Tesla.

This defect resulted in an Autopilot fatality. Now it has been fixed. Was the world made safer by rushing Autopilot to market with this defect?

No. Tesla’s per mile fatality rate without Autopilot is much lower than with Autopilot. Thus, fewer deaths would have occured has they waited before releasing. Viewed in this way, Tesla and Elon have blood on their hands.

I have 12K miles on auto-pilot and many roadtrips. If used properly with hands on the wheel as instructed everytime I turn it on and paying attention it is a very relaxing roadtripping car.

I think Mercedes uses Mobileye for its “self driving” car.

Quote: In an advertisement introducing its new E-class sedan, Mercedes-Benz touted the car as “a self-driving car from a very self-driven company.”

Four Electrics claimed:

“Tesla’s per mile fatality rate without Autopilot is much lower than with Autopilot.”

I presume this is more B.S. from this serial Tesla basher.

If there are any facts supporting that statement, please provide them. Otherwise, it’s yet another case of anti-Tesla FUDsters committing fraud on an almost daily basis here.

There is no fact, extrapolation of fact, or statistics, that support his statement.
As per usual.

Are there any non-Autopilot fatalities to date in a Model S? Honest question.

A few freak high speed accidents (including a stolen vehicle and a suspected intentional craft/suicide I believe), yes. And those are a few years old, so there are probably more.

You are missing the fact that as it currently sits right this second, Autopilot is vastly superior to every other manufacturers products that they sell as “non beta” versions.

You said: “Mobileye was so upset that Tesla disregarded their warnings that they terminated their relationship with Tesla.”

autonews.com said: “”Mobileye’s work with Tesla will not extend beyond the EyeQ3,”” … “Neither Mobileye nor Tesla would say which company initiated the move.”

http://www.autonews.com/article/20160726/OEM06/160729901/mobileye-says-tesla-contract-wont-be-extended%3B-tech-supplier-posts-76-

Four Electrics posted FUD:

“This defect resulted in an Autopilot fatality.”

It’s amazing how many lies serial Tesla bashers can pack into one short sentence, isn’t it?

The one fatal accident involving a Tesla car operating under Autopilot was a result of an illegal left turn by a truck driver plus inattention from the Tesla car driver.

We don’t know how many lives have been saved by Autopilot, but certainly there have been multiple claims that Autopilot saved someone’s life, and logic suggests there have been many more not widely reported.

For example:

“Vehicle Logs Confirm Tesla Autopilot Prevented Possible Pedestrian Death”

http://insideevs.com/vehicle-logs-confirm-tesla-autopilot-prevented-possible-pedestrian-death/

Transcript: Elon Musk’s press conference about Tesla Autopilot under v8.0 update [Part 1]

Transcript: Elon Musk’s press conference about Tesla Autopilot under v8.0 update [Part 2]

Transcript: Elon Musk’s press conference about Tesla Autopilot under v8.0 update [Part 3

Transcript: Elon Musk’s press conference about Tesla Autopilot under v8.0 update [Part 4]

Transcript: Elon Musk’s press conference about Tesla Autopilot under v8.0 update [Part 5]

Thanks for the links, Scott. Very informative.

I am afraid as long as people think this is anything more than an adaptive cruise control safety issues will persist.

In part 3 of Electrek’s transcript, Elon talks about Autosteer taking control of the steering to “nudge” the car back onto the road, even when Autosteer isn’t activated. While it may be more or less accurate to describe Autopilot/AutoSteer up to this point as a mere improvement on active cruise control, it’s clear that Tesla’s Autopilot/AutoSteer is moving beyond that.

There is a problem with how people perceive the evolution of self-driving cars. Far too many people seem to have this idea that there’s nothing between a fully human driven car, and a fully autonomous one. Those who have been paying attention, especially those of us familiar with at least the basics of computer programming, know this is very far from the case. Advancements toward fully autonomous cars will continue to be gradual.

Reliable, fully autonomous driving systems won’t come into being like Athena, who sprang forth fully developed from the brow of Zeus.

The issue is how far ahead and how wide the car can see. The current sensors are not adequate for anything more than adaptive cruise control. In order to have any kind or reliable autonomy the car needs to have a 3D image of the surrounding area or inch precise GPS system. Read about google self driving car on Wikipedia.

Elon claims, or at least appears to claim, that a Telsla car can paint a reliable image of what’s in front of a car which is in front of an Autopilot-equipped Tesla car, from the reflected radar images.

That is… not my understanding of how radar works, at all. A car in front of (or behind, or to the side of) a Tesla car would block the sensors from being able to see anything beyond. The idea that radar could somehow detect what’s on the other side of a solid object… well, as I see it there are three possibilities here:

1. I don’t understand what Elon said

2. Elon doesn’t understand how radar works

3. Radar is able to “see” around corners

Of the three, I seriously doubt it’s #3.

Google’s self-driving cars use a lidar scanner mounted on the roof. That’s the proper system and the proper geometric placement. I very seriously doubt that the sensor system Elon is talking about would measure up to the standard of Google’s cars, and I’d love to see a independent test comparing the two, by a qualified testing team or lab.

To me, the auto pilot feature looks like the perfect tool for governments and criminals to get rid of opponents, by remotely causing an “accident”… it must be easy to take over the control of the car by police, secret services, you name it. I’ll skip this tech as long as I can.

The genuine Tesla Tin Foil Hat accessory takes care of this problem.

“Any sufficiently advanced technology is indistinguishable from magic.”

Aha, but how will you protect against THEM taking control of a passing car and swerving it to run you down?

Best to never step outside at all.

Lots of interesting new stuff… but still no setting to prevent car from going faster than the Speed Assist/Speed Limit Warning nor a much more sensible option of a truly ‘relative’ offset to the speed limit ie ‘speed limit +/- 5%’.

Tesla’s TACC (Traffic Aware Cruise Control) does let you set speed according to the local speed limit +/- a certain number of MPH which the driver can set (see screen shot linked below).

But no, so far as I know, Autopilot won’t override the driver and refuse to let the car go faster than the speed limit, or the speed limit + X MPH.

Of course, there is the “Valet” mode, which does restrict top speed… but that’s not what you were talking about.

How does this radar fare when all cars on the road have the same radar? Is there no risk they will interfere with each other? If they pick up each other’s echoes they will produce an incorrect picture of the situation, or will they?

I suppose there’s no way to prevent 100% of all cases of sensor interference, but I think radar pulses can be coded in the same way that cell phone radio signals are. Giving a unique code to the pulses from each car’s radar will allow them to separate their own signals from others, just as cell phones can identify the broadcast signals associated with calls sent to each individual phone.

Presumably the same is true of using coded pulses for lidar.

I don’t think cell phone technology is comparable. Apart from different forms of multiplexing they sense collisions between several senders and then resend the data if a collision is detected.
When it comes to radar collisions I would think that the returned echo would/could be mangled so any digital information would then be hard to decode.

It’s interesting… and possibly troubling… to see Tesla going to such efforts to avoid using scanning lidar for active sensor scans of the environment. I see ffbj labeled this a “brute force” approach in the first comment here. That seems to be an apt description. Well, time will tell if Tesla’s complex computer analysis of radar images will work as reliably as lidar. Personally, I’m dubious. Here’s hoping Telsa can pull it off. However, it looks like the reason Tesla is avoiding use of lidar is because it’s more expensive. While it’s understandable that they don’t want to make their expensive cars even more expensive, I think the right approach would be to simply acknowledge that self-driving cars need scanning lidar, and start working on how to make those systems cheaper. Also, there’s no way to get around the fact that simple geometry demands the scanner be mounted on or above the roof. The higher the position for scanning, the better. Sensors mounted lower, below windshield level, will have their scans blocked by nearby objects… including other cars. I think it’s inevitable that truly self-driving cars will all have scanners mounted on or above the roof, as Google’s self-driving cars… Read more »

Have you worked with spinning/scanning LiDaRs before?

I work with them in research work (not automotive), cost is not the only limitation. Temperature, vibration, water, dust all impact the performance.

Judging by the way most people maintain their vehicle, I am skeptical of how well LiDAR will perform in production vehicles.

Assuming all the technical reliability issues are worked out, you still need to figure out packaging of redundant sensors into the vehicle. The roof mounts on research vehicles is an aerodynamic nightmare.

I think that the current scanning LiDAR set-ups seen on test cars is really just a simple way to test. The thing they’re really working on is the software.

Josh said: “Have you worked with spinning/scanning LiDaRs before?” I haven’t worked with them, period. I’m just going by what I read. So I appreciate your input here. (I do know enough to know that “LiDaR” is not a correct acronym; it’s lidar, LIDAR, or LiDAR.) “I work with them in research work (not automotive), cost is not the only limitation. Temperature, vibration, water, dust all impact the performance.” Some or perhaps most of the robotic vehicles competing in the Darpa Grand Challenge in 2005 used scanning lidar, didn’t they? That was run in the Mojave desert, where they certainly did have to deal with dust, temperature, and vibration. And there has been 11 years of development since then. Of course, a desert road endurance race of less than 24 hours is a far different challenge than working reliably and maintenance-free in a passenger car for years and years. But scanning lidar is an electronic system, and like all such systems, with time such systems will be developed to be more reliable, smaller, and much cheaper. I remember when videocassette recorders were new, and the manufacturers said “You may as well buy one now, because these are electro-mechanical devices, not… Read more »

NEWS ALERT

Just saw this story on BBC World News Business Report!

So the next version of Autopilot will be 10.0 right? 🙂

In all seriousness, Jay calling this AutoPilot 8.0 is confusing.

This is Teslas 8.0 version of the car operating system, not version 8.0 of AutoPilot. AutoPilot is around 1.2 right now, with the 2.0 hardware in the works.

This is mostly the fault of Tesla communications, but let’s try to to confuse more people.

Reading through all the upgrades, it looks like Tesla did a pretty good job overhauling AutoPilot getting the most out of the current sensor setup.

I am surprised they had the computing power to expand the use of radar not only for point cloud by to time series comparison. That is not a trivial change, expanding in two dimensions.

After the full release, we should hear feedback from the beta testers on how well it should work.

As far as the changes to the “alertness” of the driver, Tesla is still in middle ground. The penalty of having to stop the car to reset AutoSteer is OK to curb abusers.

My preference would be restricting use to divided highway only, which they would be certainly capable of, and the advertised use case for AutoPilot.

The observational data gathering is very interesting, whether the feature is on or off, comparing the decision making of the driver compared to the software and building the database.

I better not follow Tesla too close. Now, it is going to brake hard if the system “thinks it needs to”. Maybe it is warranted, maybe it is not. But I better keep my distance for sure.

Please do!