Consumer Reports Tests Tesla’s Navigate On Autopilot

NOV 6 2018 BY MARK KANE 33

Tesla Navigate On Autopilot requires improvement to be better than drivers.

Consumer Reports tested the latest Tesla Navigate On Autopilot feature near its Auto Test Center in Colchester, Connecticut and is impressed with its findings.

However, the ability to do route-based lane changes or speed-based lane changes on the highway (as well as take an exit off the highway) was called more like a glimpse of future self-driving cars than a ready product (it’s a beta release).

The Navigate On Autopilot brings not only a promise but also a new set of problems and some concerning limitations.

Consumer Reports noted:

  • After speed-based lane change, the car stays in the left lane (what we saw from the beginning in many videos posted by users), which without driver intervention will turn Teslas into “the world’s worst left-lane hog”. It should be an easy fix for the next update we believe.
  • The next problem is that sometimes the car would like to change lanes despite there being a faster-moving vehicle coming up alongside, that would be cut off. According to the article, Tesla intends to release an update that will use three rear-facing cameras to detect fast-approaching objects “better than the average driver.” But, as we understand, better than average is not necessarily as good as a good driver (time will tell).
  • Test drivers noticed also that the Navigate On Autopilot works best in situations with minimal traffic and struggles in heavy traffic.

    “As the system currently stands, CR’s testers found it lagged behind their own skills while driving in heavy traffic. “Overall it works best in easy situations with minimal traffic,” Fisher says. “But as the traffic builds, it clearly displays the limitations of today’s technology.”

  • The last concern is highway exits when you drive behind a slower vehicle. CR experienced a situation when Navigate On Autopilot decided to overtake the truck, taking the left lane not far ahead of the exit, and because there was heavy traffic, later the Tesla needed to slow down. Normally, the driver would stay behind the truck and wait a while to take the exit smoothly and safely. Foresight is missing.

    “The system also had issues navigating interchanges: Not far before exiting a congested highway, the system recommended that the driver change lanes to pass a slower-moving truck instead of waiting to exit as most drivers would choose. But traffic was too heavy for the Tesla to complete the pass. Autopilot slowed the car considerably so it could drop back behind the truck and into the exit lane—which slowed traffic in the process.”

    “Tesla told us that its feature relies on many factors, including “the speed of the cars in your lane, the speed of the cars in the adjacent lane, the density of the cars in the adjacent lane, and the amount of time the car has to complete the lane change before the next route transition.” In certain “unlikely” situations, it might even slow the car to a complete stop, the Tesla spokesperson said.”

    “It was able to execute the exit, but it wasn’t pretty,” Fisher says. “It didn’t have the foresight to say, ‘With all this traffic, passing so close to the exit is not a smart thing.”

In other words, technically we are probably still at least a couple updates before autonomous driving could be possible in the base case scenario – on the highway and in good weather.

Source: Consumer Reports

Categories: Tesla

Tags: ,

Leave a Reply

33 Comments on "Consumer Reports Tests Tesla’s Navigate On Autopilot"

newest oldest most voted
GSP

Great input from CR. People need to know what Autopilot can do and what it can not (yet).

I am looking forward to getting the Navigate on Autopilot software for my car. I currently have version 9 which does use all 8 cameras, but without Nav on Autopilot. Ver 9 does a much better job of changing lanes than ver 8.1. The lane changes are super smooth, much better than what I would do manually. It also can speed up or slow down to line up to a space in the new lane. It is a joy to drive.

However, more than few updates to address CR’s concerns will be needed for autonomous driving (level 3 or 4). For one thing, obstacle and chuckhole detection and avoidance will be needed. Autopilot will still be level 2 (driver must always supervise carefully and take control when needed) at least until Tesla’s new HW 3 computer is available.

GSP

Else

Surprised they didn’t make any references to Cadillac’s

Allan

I don’t think the cadillac matches these capabilities on a technical level, but CR did give it a higher score because it’s better at watching the attentiveness of the driver iirc.

Rightofthepeople

I agree with CR. Took a 1k mile road trip in the 3 this weekend and tested Nav on AP. Passing recommendations were sometimes quirky but executed smoothly. Definitely needs to check upcoming traffic better before making the decision to pass. I was impressed with the auto exit from one freeway to another. Progress has been made and much more will be made.

J. L. Brown
This is sort of a ‘nothing sandwich’ from the Consumer Reports people. They took the earliest release available and found some edges and corners — shocking news, this ain’t. After the repeated-emergency-braking bruhaha, I would think that CR would be a little more cognizant of how quickly progress and OTA updates can improve features of Tesla vehicles. More importantly, Consumer reports rates and ranks lots of products — and their most recent comparison of AutoPilot rated it slightly below the Cadillac SuperCruise; how does this new feature affect that ranking? Does any of this merit a mention in their evaluation? Apparently not. From the highway-interchanges criticism: ‘Not far before exiting a congested highway, the system recommended that the driver change lanes to pass a slower-moving truck instead of waiting to exit as most drivers would choose.’ AND YOU LET IT?!? WHAT THE ACTUAL F***!?! The system relies on oversight from a more-competent human, that is why it is a driver-assist feature! Worse, this bad call on the humans part is now part of the training data for the neural net! Okay, fine, I trust Tesla to vet the data to edit out boneheaded moves like this — but it was… Read more »
Mahesvara

lol never trust Consumer Report as they aren’t the representative of all consumer in US…………… it is self opinion like what others do…………

Else

They represent Toyota and Honda. Notice how silent they are on Toyota and Honda,’s autopilot efforts or lack there of?
Imagine if Toyota, Honda, GM had a system as capable as Tesla’s, they would be praising it all day long.

Viking79

They definitely don’t represent Toyota and Honda. They represent their members, and their members value certain things in a vehicle like reliability and ease of use, and what those members like gets reflected on the cars. Honda and Toyota design their cars to the needs of what their customers want and a typical CR member is maybe their target audience, looking for a reliable safe vehicle.

CR advocates for consumers, so they see self driving as a place where there are going to be numerous oversights in safety from companies looking to monetize on self driving immediately. It is a bit of a wild west zone and they want to minimize that. So yes, they are talking about Tesla here, but that isn’t necessarily a bad thing.

Else

Horses and carriages we’re also reliable!

Cypress

Tesla probably wouldn’t have pushed an update so fast to fix the braking issue, if CR hadn’t done independent testing and called them out for the issue.

Else

How many issues GM, Toyota,Honda have and CR don’t criticize them?, they give them a pass. I am sure CR brought it up because they thought Tesla would emit a recall. But to their surprise, Tesla did an OTA update to fix ” issue”

ffbj

..and their amazement was great as they correctly pointed out no else is even capable of doing this, and probably won’t be for many years.

carcus

… and yet, we’ve been hearing for some time now (a few years?? ) how self driving cars are able to successfully navigate from coast to coast … only carrying a human along to serve as a safety observer.

R.S

Maybe they don’t have that much issues? Look, it’s a cool thing, that Tesla can do OTA updates, but in general it’s not good if you buy a car that needs an update to work safely and reliably.

Now if CR can find faults and Tesla solves them quickly over the air, the owners benefit from it.

Pushmi-Pullyu

“This is sort of a ‘nothing sandwich’ from the Consumer Reports people.”

I’ve been fairly critical of how CR has handled its Tesla reviews in the past, particularly in how arbitrary and apparently erratic the reviews have been, but here I think they’re providing a valuable service. Yes, it’s true that this is the first release of an upgrade which is still in the “Beta” stage, but I think it’s quite useful to have a review very soon after release that tells us what the system can do well, and where it needs improvement.

And as another comment already pointed out, this is something that no other auto maker’s cars do, no matter how poorly. To say that CR’s report is a “nothing sandwich” because Tesla will undoubtedly improve things before long, is IMHO really missing the point. CR is saying “Navigate on Autopilot” isn’t perfect, but that’s very different from saying it’s not a significant advance. CR is giving this upgrade a review precisely because it is a significant advance.

Vexar

This is probably the most useful thing Consumer Reports has done for Tesla: a balanced review. There are a fair number of Tesla owners who will discover this for themselves, but there’s this older, newsprint generation that finds reading things more informative than exploring and learning on their own. I hope they continue to do reviews at major release points with the autopilot. Who knows? They might become the de-facto standard for reviewing autonomous vehicles if they really push hard into this, and learn about artificial intelligence, neural networks, expert systems, LIDAR, machine vision, and so forth.

Mahesvara

lol when you put Cadillac driver assist above Autopilot then you have a problem…………….

Else

Every single owner knew that they system is in Beta, but compare to before, it is great. Tesla is not stopping there, but the developers need encouragement.

I for one hate when the system tries to center itself in the lines during s merging situation. I think it’s dangerous and scary, it should hug the closest line to it until after the merging or on ramp situation

Sidney Reilly

By the sounds of it, the car will be better than 90% of the drivers in Colorado where left-lane hogs are the norm and drivers are so busy holding a phone to their heads that getting on or off the freeway is of secondary importance.

Cypress

So when is Tesla going to stop calling AP a beta product? Or is there some legal liability avoidance they get for calling it a beta product? Will it be in beta indefinitely?

Else

I rather have it now and see it grows than waiting 15 or 20 years when it is matured

PC

I’ll start off by saying that I’m a huge Tesla fan and have used Autopilot (with proper care) on many trips for 90+ percent of the drive. That said while it seems to me that CR has a negative bias against Tesla they did a good job reporting on the latest Nav / Autopilot release and what they said is accurate. But that is what is to be expected in beta software and as is mentioned in the other comments a combination of Autopilot and organic processing is currently the best solution. Also mentioned is the value of OTA updates and how it will improve the functionality. That and the increased processing power (hopefully) coming next year.

PC

Else

They could have said” Tesla released a great update to Autopilot,but there is more to come”. For instance, during this and that the car did not perform well, in other situation , it should fo this instead of that.
” Overall, progress is been made”

Chacama

” the car stays in the left lane (what we saw from the beginning in many videos posted by users)”. I for one have the exact opposite results: Left lane for passing only as the car always wants to return to the right lane even if there is no cars in front or behind.

Alex

Humans are still a lot smarter than self driving systems… for now. From reading TMC forum it seems that this feature is a stress maker – it will do something silly at any moment and driver must be very aware, rather than a stress relief.
Per example, even the lane keeping is just silly in a road without traffic, I want to “invade” other lanes to smooth out curves (even if to be rigorous that’s against the rules) it makes a much more pleasant trip, it doesn’t bother me much having to slightly turn the wheel.

MikeM

@Alex: “Per example, even the lane keeping is just silly in a road without traffic”

Interestingly (to me) I’ve found that, after resuming human control, for the first few minutes I tend to maintain the light steering wheel touch to the point that I occasionally forget and Iet the car drift onto/over the line. And I mean way more than might happen in normal driving. That takes me from the realm of “just silly” to “just dangerous”.

The whole autotpilot experience feels a bit like having Fido do cute doggy tricks except he’s a fast 4000# death dealing missile with me riding along!

Cypress

LOL!

Quin

That’s why the lane changes are suggestions, if the change isn’t a good idea, don’t accept it.

That said, NoA does have limitations and isn’t full self driving on freeways yet.

Roy_H

And I expect it still ignores stationary objects like cement barriers, fire trucks, tow trucks etc. Apparently this is now acceptable and not even mentioned.

Pushmi-Pullyu

You have clearly confused what Tesla calls “Navigate on Autopilot” with what Tesla calls “Full Self-Driving”.

Fortunately, Tesla has not confused the two.

Will

Go out and vote

CCIE

It’s cool tech. though still very much beta. Now if Tesla could just master all the basics, like providing reliable/consistent/safe regen on RWD M3s when using winter tires. Based on early reports from Canada, we’re going to see a lot of accidents when people in the US snow belt install winter tires and have to deal with erratic regen.

Vinay R

Navigation on Auto pilot is at very early stage, Right now it’s suggesting to change lane based on merging lane coming ahead and car slow down in front, most of the suggested lane change is not how most people drive, if there is no traffic in upcoming merging lane you don’t change lane but It’s suggesting to change lane, don’t know how Tesla will tackle this problem, perhaps through maps data in coming software updates.