See What Tesla Autopilot Sees In Downtown Paris Drive

SEP 25 2018 BY STEVEN LOVEDAY 11

Here’s what Tesla Autopilot “sees” during a drive in the Paris city limits.

We’ve shared videos with you before about what users learned that Tesla Autopilot is “seeing.” However, a drive around downtown Paris is quite a bit more interesting than just any random road. Of course, this data doesn’t come from Tesla, nor did anyone at the automaker help to reconstruct it. Instead, the owner has taken the time to use internal data about what Autopilot sees and then reconstructed the video to paint a picture for us.

This footage only applies to the front-facing camera. Being that it “sees” a lot, it would be very telling to learn what all the other cameras are taking in and processing. While this gives us a good idea of how Full Self-Driving could eventually come to fruition, it also reveals the ultimate complexities of the situation. Nevertheless, it’s pretty neat to see it in this setting.

Video Description via greentheonly on YouTube:

Paris streets in the eyes of Tesla Autopilot

This is what Tesla autopilot “sees” via the ‘main’ camera when driving on the streets of Paris. Objects, lanes, drivable space (green fill at the bottom), ….

Footage is reconstructed from the autopilot videofeed and a stream of internal data. This footage was created without any help from Tesla.

Timing highlights:
01:17 – traffic cones shape driveable space
01:31 – construction equipment recognized as a truck (shows they have quite a deep library of objects they train against? Though it’s not perfect, we saw some common objects not detected too. Notably a pedestrian pushing a cart (not present in this video)
02:23 – false positive, a container mistaken for a vehicle
03:31 – a pedestrian in red(dish?) jacket is not detected at all. (note to self, don’t wear red jackets)
04:12 – one example of lines showing right turn while there are no road markings of it
05:52 – another false positive – poster mistaken for a pedestrian
08:10 – another more prominent example of showing left turn lane with no actual road markings.
09:25 – close up cyclist
11:44 – roller skater
14:00 – we nearly got into accident with that car on the left. AP did not warn
19:48 – 20 pedestrians at once (not that there was shortage of them before of course)

Categories: Tesla, Videos

Tags:

Leave a Reply

11 Comments on "See What Tesla Autopilot Sees In Downtown Paris Drive"

newest oldest most voted

Like in Terminator first movie:)

Good to see Autopilot is reasonably reliable in spotting parked cars, altho I did notice at least one instance where it grouped two parked motorcycles into a single object. But this is all low speed driving; I presume that such observation of parked vehicles is shut off at highway speed, or at least it’s not very reliable. If it did work reliably at highway speed, then Tesla cars controlled by Autopilot +AutoSteer would not keep running into fire trucks parked on the highway.

I note that Autopilot is taking no notice whatsoever of traffic lights. I think Tesla has shown a video of a more advanced version of Autopilot which does “see” and obey traffic lights, but it looks like the mass produced version installed in cars isn’t even observing traffic lights in “shadow mode”.

Maybe because they are French stop lights.

And in France red traffic lights are only proposals ……

How often did that happen with Autopilot actually engaged? And more importantly: when was the last time it happened with Autopilot engaged? If memory servers right, that was quite a while ago — long enough that it doesn’t really make sense to try correlating it with the data presented here from the current software version…

” If it did work reliably at highway speed, then Tesla cars controlled by Autopilot +AutoSteer would not keep running into fire trucks parked on the highway.”

You know very well that the Autopilot is simply not programmed to stop for stationary vehicles on the highway. It’s been discussed here ad nauseam.

It’s impressive, but also shows just how complicated the real world is. The trouble is you can have a huge hit rate, but all it takes is one miss and you can have a bad result.

looks like AP1 mobileye cameras, scene looks very similar to 3-4 years ago processing capabilities from their presentations

What struck me is that the Autopilot does not scan for traffic lights. (which was already noted by Pushmi I see).

And at 4:53 the driver (Autopilot?) should have yielded to the pedestrian.

I’m working on deep learning computer vision for medical R&D as opposed to automated driving, but this is so impressive! The real-time accuracy of finding the bounding boxes for cars/bikes/pedestrians/moto is really spot on, the segmentation of the road area is consistent and accurate, and the lane finding is great. Really cool to see all of these technologies come together, and shows how they are doing it without LIDAR. It only gets better from here!

To build confidence in the system, Tesla should have an option to turn this view on in the car. Maybe it isn’t as good as we expect, but if it is, then seeing what the computer is processing would help acceptance and building confidence. I think I saw Waymo does it that way to build confidence.