Home
Tesla
News
Watch Tesla Auto Labeling Tool Working To Understand What It Does
It’s currently not running in actual cars, but the plan is to use it to sift through FSD video footage.

Andrei Nedelea 
By: Andrei Nedelea
Dec 3, 2021 at 1:02pm ET
Add InsideEVs as a preferred source in Google
- (https://twitter.com/intent/tweet?url=https%3A%2F%2Finsideevs.com%2Fnews%2F552340%2Ftesla-auto-labeling-tool-video%2F&text=Watch%20Tesla%20Auto%20Labeling%20Tool%20Working%20To%20Understand%20What%20It%20Does&via=InsideEVs)
- copy
Share
Comment
Tesla has a lot of footage collected by vehicles included in the Full Self-Driving program, but it's a lot of work to go through all that manually. Right now, the work of identifying and labeling cars, people, other road users, traffic signs and pretty much anything else that a car might come across on the road is being done by hand.
But Tesla is working on a tool to automate this gargantuan task, get AI to do it and it already looks like it’s doing what it is supposed to. Videos showing an early version of the auto-labeling tool working was shown by the company’s AI boss, Andrej Karpathy, on Twitter and if you understand what’s happening, it’s pretty fascinating to watch.
The software is not yet running in vehicles, but you can see it very easily identifies primarily pedestrians and other cars - even cars parked in lots quite far from the road, which are only visible to the vehicle’s side-facing cameras, are clearly marked with blue boxes.
What do you think?
View
Comments
Road signs are marked in green, traffic lights are orange and you can see the AI is able to clearly define where the sidewalk is (marked in red) and the actual road (which gets a slight purple hue). The system also seems to use teal to mark the edge of what the car can drive on and it also appears able to detect if a surface is paved or not - it shows the unpaved area around trees on the sidewalk as green, as well as other unpaved areas.
Karpathy’s description for the two videos that he posted makes it seem harder to understand what’s happening, but this tool does what its name says - it automatically labels. The lingo used in the description seems aimed at people who are familiar with neural networks and artificial intelligence, so perhaps Tesla is looking to attract new talent to help get this tool implemented and working.
More self-driving cars
2022 WM Motors M7 EV Has Hardware For Level 5 Self-Driving
XPeng's New 'Smart Driving Points' Mirrors Tesla's Safey Score
Source: Andrej Karpathy / Twitter
We want your opinion!
What would you like to see on Insideevs.com?
Take our 3 minute survey.
- The InsideEVs team
Share this Story
- (https://twitter.com/intent/tweet?url=https%3A%2F%2Finsideevs.com%2Fnews%2F552340%2Ftesla-auto-labeling-tool-video%2F&text=Watch%20Tesla%20Auto%20Labeling%20Tool%20Working%20To%20Understand%20What%20It%20Does&via=InsideEVs)
- copy
GO TO COMMENTS
(
)
Got a tip for us? Email: tips@insideevs.com
View all comments ( )
Trending

I Tried Mercedes' Answer To Tesla FSD. Here's How It Went

Driven: BMW's Electric M3 Is The Real Deal

Tesla Opens Self-Driving Cybercab Orders To 'Select Customers' At $29,990

Tesla Model 3 Maintenance Guide: What Needs Servicing And What Doesn't
Latest Stories
- The Electric Baby Land Rover Defender And Range Rover GT Will Begin Production Next Year Nov 16 - News -
- Mercedes Is Bringing Hands-Free Driving To Certain U.S. Highways In 2027 Nov 16 - News -
- Tesla Keeps Killing Cars. Now It Wants To Build Cybercabs In Germany Nov 16 - News -
- Tesla Model 3 Maintenance Guide: What Needs Servicing And What Doesn't Nov 16 - Repair / Maintenance -
- Audi A2 e-tron First Drive: Does The Iconic Nameplate Live Up To The Hype? Nov 16 - First Drives -
- I Tried Mercedes' Answer To Tesla FSD. Here's How It Went Nov 16 - Tech Review -
- BMW Just Started Making EV Batteries In America. The New iX5 Will Be First In Line Nov 16 - Manufacturing / Production -
RECOMMENDED FOR YOU

Tesla Keeps Killing Cars. Now It Wants To Build Cybercabs In Germany

Audi A2 e-tron First Drive: Does The Iconic Nameplate Live Up To The Hype?

Tesla Model 3 Maintenance Guide: What Needs Servicing And What Doesn't

I Tried Mercedes' Answer To Tesla FSD. Here's How It Went

Tesla Just Bumped The Cheapest Model Y’s Range, But The Price Stays The Same

BMW Just Started Making EV Batteries In America. The New iX5 Will Be First In Line

Tesla Drivers Can Now Plug In And Charge Automatically at Ionna Stations