What exactly is Tesla's Autopilot system "seeing."

This is not the first time we've seen and shared what Tesla Autopilot cameras "see." However, this newest video takes it to another level, with interpretations of how the technology is processing and digesting the data it receives. Additionally, the creator of this video includes some clever assumptions about the system, which help to make it easier to understand the visuals.

Tesla shared early videos with recognition feed not long after it debuted its Full Self-Driving technology. However, the automaker hasn't carried out its self-driving coast-to-coast trip and the software is not yet available. Since the automaker's initial wave of videos, we haven't received much more information.

Thanks to the hacker community, we now have more detailed insight.

Video Description via greentheonly on YouTube:

Overlaid with some data in addition to the usual circles:


Color of the circle represents the type: green - moving, yellow - stopped, orange - stationary, red unknown.


Size of the circle represents the distance to the object (NOT THE SIZE OF THE OBJECT). If my assumptions are correct you should imagine that the circle has about 1.5~2m diameter in 3D space in place of the object (but it does not get smaller than 100 pixels).


When it is drawn with thicker line that means it has a label and most likely the object is being tracked. When it fades in and out it is caused by changing probability of existence.

Source: Electrek