New Tesla Autopilot To Operate In “Shadow Mode” Until Self-Driving System Is Ready To Activate

OCT 23 2016 BY MARK KANE 37

Tesla's new self driving hardware/360 degree vision to allow for Level 5 autonomy

Tesla’s new self driving hardware/360 degree vision to allow for Level 5 autonomy…someday

Tesla Motors recent announcement about installing full self-driving hardware into today’s vehicles to one day provide Level 5 autonomy is a big news, however its not likely we will see such a capability for the casual driver anytime soon.

Newly equipped hardware to enable future Level 5 autonomy - like this new side camera

Newly equipped hardware to enable future Level 5 autonomy – like this new side camera

Looking back at recent history, a year elapsed between when the first version of the hardware was delivered in production cars, and the actual activation of the Autopilot.

The story looks to again repeat (and then some), especially in that Tesla needs to not only develop the software to be able make use of the hardware in cars, but also get permission to actually then activate the system in various countries.

The latter (regulations) most likely being the most time consuming process, as places such as the US don’t appear to be anywhere near signing off on such an initiative at this point.

This is why the hardware will mostly operate in some kind of Shadow Mode, collecting piles of data that later the company will make use of – checking whether its on board computer and subsequent software can cope with all the particular situations it is presented with on the road.

Says Tesla of its new system:

Eight surround cameras provide 360 degree visibility around the car at up to 250 meters of range. Twelve updated ultrasonic sensors complement this vision, allowing for detection of both hard and soft objects at nearly twice the distance of the prior system. A forward-facing radar with enhanced processing provides additional data about the world on a redundant wavelength, capable of seeing through heavy rain, fog, dust and even the car ahead.

To make sense of all of this data, a new onboard computer with more than 40 times the computing power of the previous generation runs the new Tesla-developed neural net for vision, sonar and radar processing software.

Several new cameras are part of the new fully self-driving hardware upgrade

Several new cameras are part of the new fully self-driving hardware upgrade

With so many working electric vehicles almost immediately returning data in the vehicles today, there will be a lot of data for Tesla to work worth, analyze and sort into some form of logic database.

Indeed, the mountains of statistical data collected from today’s Model S and Model X (as well as the upcoming Model 3) could be then used to accelerate the necessary permissions to use the system on public roads.  Judging by recent reports, Tesla will add about 2,000 EVs with the new hardware every week to its data collecting fleet, so we are eager to get first insights back from the Borg collective.

Says Teslarati on the data collecting:

Even though enhanced AP costs $5,000 and will be active in December, "Full Self-Driving Capability" will set one back a further $3,000...with no expected ETA

Enhanced AP from Tesla is a $5,000 option and will be active in December.  However, “Full Self-Driving Capability” will set one back a further $3,000…with no expected ETA

“New Tesla Model S and Model X automobiles will run Autopilot in “shadow mode” and collect driving data that pits a human versus computer. Autopilot vehicles running in shadow mode will not take any driving-assist or self-driving actions. Rather they will only log instances when Autopilot would have taken action and compare those results to the real life actions taken by human drivers. Musk told the press that the ultimate goal is to improve its self-driving algorithms until they are better than human drivers. By having statistical data to back up the safety of its self-driving model, Tesla will have a better chance of proving to regulators that its vision for a Tesla-powered autonomous future will be safer for humanity.”

“Musk said that it would be some time before Tesla’s software would advance to meet capabilities of Hardware 2, so it will be disabled until it “reaches parity following full validation with Hardware 1, probably in December.””

Also of interest:  Check out a Tesla Model X showing off its recently added autonomous abilities using the new hardware in a Tesla produced/controlled video demo here.

source: Teslarati

Categories: Tesla

Tags: , , ,

Leave a Reply

37 Comments on "New Tesla Autopilot To Operate In “Shadow Mode” Until Self-Driving System Is Ready To Activate"

newest oldest most voted

Tesla will clock 1 Billion miles by the end of 2017 and 9 Billion miles by the end of 2018. That is enough data for level 5. Let us see what GM etc. will do about that? Even Nissan can’t come near.

Don’t forget GM has OnStar installed in just about every GM vehicle now. So they are collecting telematics and sensors data on 9 million additional new vehicles annually. The data might not be as deep as Tesla, but they would have a lot more of it.

Why is it that they don’t have a level 5 by today then?

“Deep” data is the most important…OnStar is basically just GPS data and accident notification…It has no way of knowing after you crashed if you were texting and driving or if someone came into your lane…

These sensors generate ~1 T of raw data daily. How does Tesla upload all that? Answer, they don’t. At best they upload a few seconds worth when the driver and software disagree. Google, Uber, GM/Cruise, etc. drive a lot fewer miles but collect 100-1000x more data per mile.

Both approaches have pros and cons, my point is you can’t just blindly compare “miles”.

But you are making an assumption that Tesla does not have its own fleet (similar to google/uber) for large data collection.

Mine is hooked up to WiFi (1 GB/s) every day… not sure why they wouldn’t be able to download a few TB. Obviously not for all… but still.

I fear that what will happen with self-driving technology is that the driver will lose interest in monitoring the car as it drives itself. They will become distracted or even take a short nap.

Self-driving means that you could and should take a nap. You are not the driver in a self-driving car.

With a driver assist system like the “autopilot” in the other hand…

Level 5 means that the car can drive solo and knows how to behave if something brakes down (Camera defect, Computer chip fail, etc.).

There is no risk in being a unatended human as this is the purpose of Level 5 autonomous driving.

Actually that’s Level 4. Level 5 (unofficially) means no steering wheel or other driver controls.

That’s the plan indeed. Which could quite hazardous but if proven fully finctionavlr and up to standards or even better why not?

in other words, if you buy a tesla, your driving habits will be monitored in great detail. i’m surprised that we haven’t seen a privacy lawsuit already. at least when you bought a chevrolet Volt, you have to “opt in” for the free on-star service period. when you decided to “opt in”, you gave gm permission to collect information about your use of the car.

How cute, you think Tesla cares about “no comments” driving behavior…They want to improve algorithms using complied data…

The Volt, like most if not all new vehicles, have a “black box” data recorder…Authorities (or hackers) can access your cell phone including GPS anyways…

I’d rather see the pricing be split, 4k and 4k….wojld make the enchanted AP a bit more affordable for lower end buyers. But I guess they figure if ppl will pay 5k, what’s another 3k to get the full feature set?

I believe that it have to do with the cost as they install all hardware in all cars regardless what the customer order. In this way I believe that most people will at least buy the package for 5k wish means higher income fot Tesla.
Earlier this year Nvidia sold the ship for developing programs 15k, Tesla probably don’t pay close to this sum but it’s not free.

I am so incredibly happy to see Tesla going for level 5 autonomy so quickly. I’m a huge critic of driver assistance systems as agents of distracted driving, but level 5 is the holy grail. It may finally make bicycling on public roads safe, and usher in a revolution in how we use and park cars. More power to them!


It is going to take years to get permission for such a system. At least they can argue that google already has a computer licensed as driver.

I don’t understand the celebrations. AP was deeply flawed, even at the relatively simple task of separated highway driving, even after 8.0. This is a many orders of magnitude more complex system, let’s wait and see if it really delivers or if it’s just more hype.

If you have ever used the current AP system for any time at all you will understand how massively incorrect your statement is.

So autopilot worked well in rain, fog, snow residual streets, etc..?

Well think of it this way. With all the new extra hardware being added AP will function like it should. So when you pay that $5,000 extra for your Model 3 you will get fully functioning AP that is better than what is in the current $80,000 Model S. That’s the bright side.

Tesla’s approach to Level 5 is the correct one because it is based on deep learning that will continue to improve as more and more miles accrue.

It might start off as Level 3 but it will continue up the ladder rapidly and get there first as a commercially viable system.

Once it does then it will have a MASSIVE market for taxis and other light duty fleet operators because combined with the much lower TCO of a BEV it will quickly start replacing all those vehicles that operate at a high number of miles per year.

When the Tesla Semi Truck comes online it will likewise have the same effect there as it will on the light duty fleet.

I think one of the issues with Tesla’s approach is deep learning. In the end this is an AI issue. Yes deep learning is very important, but at some point you have to have an intelligence to deal with a new situation.

If an actual intelligence was necessary for driving a car; that is, a self-aware sentient mind, then we might as well forget about self-driving cars. We are very far away indeed from true machine intelligence, actual A.I.*, despite what many science fiction movies and TV shows portray.

But I’m confident that this won’t be necessary. Some of Google’s newer self-driving cars don’t even have a steering wheel or pedals. This wouldn’t be possible if actual A.I. was required, even in rare cases.

*Unfortunately the term “A.I.” (Artificial Intelligence) has been misused by advertisers to mean merely software which functions as an expert system. This is wildly abusing the term. There isn’t a computer system in existence that actually demonstrates thinking at a level more sophisticated than an insect.

Question is, what happens if a police officer wants to stop an autonomous vehicle? Who is at fault if the car is speeding (perhaps the car missed the speed sign)?

There will be laws regarding the matter. Cops won’t need to pull over an autonomous car. If you are asking about the physical feasibility, it won’t be much a deal for the rear cameras to pick up police beacon lights and pull over.

I would like to see an autonomous override that law enforcement could use. Just type in the license plate and it would give them the ability to simply command the car to pull over at the nearest safest location without the driver being able to override. How many people have been killed or hurt by idiots running from the cops while being pulled over or after they are pulled over and decide to run.

It will also take away the scenario where a police officer has to decide whether or not to shoot at a suspect driving at or near them.

The down side is they will have to cancel all those shows like “wildest police chases”.

There are a trillion “what if?” questions that regulations will ultimately find answers for…

“Someone out there” asked: “Question is, what happens if a police officer wants to stop an autonomous vehicle?” Well, how will autonomous vehicles be controlled? Generally they will have at least one passenger who can give instructions to the car. If the passenger hears a siren or sees an emergency vehicle’s “bubble gum” lights flashing, then he can instruct the car to pull over safely. But I envision a future in which autonomous cars will wirelessly communicate with each other and/or with local traffic computers; possibly control “nodes” located like cell phone towers. With the cars interacting with each other and/or with a central traffic control computer, cars can be instructed to get out of the way of emergency vehicles and/or instructed to stop if a policeman wants to perform a traffic stop. Note that in my scenario, it would never be permitted for an autonomous vehicle to drive on public streets without having either a passenger or being able to wirelessly communicate with other vehicles. The latter would, of course, give priority and right-of-way to emergency vehicles. “Who is at fault if the car is speeding (perhaps the car missed the speed sign)?” This is something that will have… Read more »

Meanwhile a tour bus smashes into a semi at high speed killing a number of people and seriously injuring many others, most sleeping and not belted into their seats. The driver was probably sleeping too.

Yes, this scenario is probably the reason buses will be the last to implement full AP. I wonder how many years it will take.

I so wish that this whole self driving thing could take a break. We first need the companies to focus on delivering electric cars (and lots of them). Looks like this whole autonomous thing is creating more of a distraction. Its a good thing, it should come. But one of the reason we don’t have too many electric cars is because they are expensive. Model 3 is supposed to change that in theory, but adding all this autonomous stuff (hardware, research/software) cannot be for free and should (and probably would) add to the bottom line cost. So it would have been better to focus on getting the cheapest cars out first to increase adoption. I also worry that hope this autonomous thing will not cause any further delays. Almost everyone I talk to would love to buy a Tesla if it was cheaper.

Think of it the other way, that AP improves EV adoption…From day one it was announced that the TM3 would get AP and it’s NOT allowing fully autonomous at launch so that can’t really delay it…Hardware is surprisingly cheap…But a lot of people want this and even if they refuse to let the vehicle drive them there’s opportunities for the car to drop you off at your doorstep, go charge itself and park it after…

One interesting thing I’ve been reading about are the self-driving (with driver backup) for the Pittsburg Ubers…At least one accident and a car being caught going the wrong way down an one way street…Uber said it cannot drive over bridges and expects a lot of issues when it snows…They also said they’re having difficult with “disappearing” objects such as a mother duck and her ducklings crossing the street…Have to program the car to stop and wait vs running them over…

Yeah. You bring up several points which are among the reasons why I say that despite what Elon is claiming, there is no way Tesla — or any other auto maker — is going to start selling fully autonomous cars within the next two years. Even Google says their self-driving cars can’t deal with snow covering the roads.

My prediction is that for at least the next 10 years, the most we’ll see is cars which are self-driving only under certain conditions. Under other conditions, the car will turn the responsibility of driving to the human sitting in the driver’s seat. That’s not what I would call “full autonomy”, but maybe Elon will call it that.

Cars without steering wheels sold to the general public? I just don’t see that happening within the next 10 years. Maybe not within 20 years.

Self-driving cars are not going to suddenly arrive on the market fully developed, like Athena springing fully grown from Zeus’ brow. They are going to evolve gradually, just as they’re already doing.