After we wrote about a Tesla Model Y with Luminar Hydra LiDARs in tests, company advocates quickly pointed out that Tesla has used them for a long time. There are indeed images of Tesla prototypes with massive LiDARs around. Yet, Tesla could also be evaluating their use, which would be no issue at all if had not said it is a “fool’s errand.” We talked to specialists to understand the implications of these two possibilities.
Relying solely on cameras would make Teslas more affordable, but the main question is how much more. On the other hand, suddenly incorporating LiDARs would disappoint all customers who trusted Teslas would have “all the hardware necessary, compute and otherwise, for full self-driving,” as Musk said on April 22, 2019, at the Tesla Autonomy Day. Some have already been let down when they received that, in Musk’s words, would make full autonomy possible.
To make matters even more intricate, Al Prescott, the Tesla VP who said MCU units in the Model S and Model X were wear parts, left Tesla to join Luminar on April 21, 2021. Although this could just be a coincidence – apart from being pretty ironic – it is one that we would fail not to mention.
InsideEVs got the chance to talk to Algolux and AEye, two companies that – like Tesla – focus on cameras associated with autonomous driving technologies. While Algolux aims to optimize camera-based perception systems, AEye is developing iDAR, a piece of equipment that congregates both LiDARs and cameras in a single unit.
Dave Tokic is Algolux’s VP for marketing and strategic partnerships. We asked him if it made sense that Tesla used LiDARs for training. Although Tokic thinks the strategy is plausible, he thinks Tesla needs to perfect it.
“For perception tasks, such as driver warning through automated functions (AEB, highway pilot, self-parking...), understanding accurate distance is critical. If the car is going 50 km/h (~14 m/sec) and a person/vehicle is 30 m away, it implies a different response (emergency brake immediately!) than if they are 100 m away (no need to react). Usually, these functions are primarily radar-based but this is also done with cameras. With stereo cameras, you can get rather accurate distance estimation geometrically depending on the baseline – the distance between the two cameras. Most ADAS stereo cameras are short baseline (~20 cm). Subaru's EyeSight is a well-known example of wide baseline (~50 cm) that provides more distance estimation accuracy. Clearly, there are issues with Tesla's system if they continue to run into trailers.”
The problem is that having cameras do all the work may not be the most effective or safe option, according to Ove Salomonsson. He is the senior director of product quality and functional safety at AEye. Salomonsson also agrees that you can use LiDARs to improve camera accuracy.
“LiDAR provides ‘ground-truthing,’ a true 3D depth measurement, whereas cameras are only capable of interpreting 3D depth based on a combination of 2D data and clever computer vision algorithms that learn to interpret depth based on training data that combines camera and LiDAR data. This reinforced approach to training teaches the onboard computer to learn ‘what depth looks like’ by relying on accurate, non-interpretive sensors such as LiDAR. The downside is that this is still a computational process versus having precise measurement in 3D from the LiDAR sensor itself.”
In other words, computers interpret the images generated by the cameras and estimate the distances. LiDAR really measures it, as Tokic stresses.
“Validating a depth system to correlate how accurately it measures depth versus some ground truth is done in a few ways, such as pre-measured targets on some closed/controlled site and using LiDAR for dynamic measurements. Reasonable they would use LiDAR for that, especially for single or short-baseline stereo camera depth sensing.”
If LiDAR was used only for training purposes, the Tesla Model Y photographed in Palm Beach would be just a “labeling car.” Salomonsson told us more about them.
“Labeling, also known as ‘human-verified labeling,’ is when a human annotates (outlines and highlights objects of interest) in a frame of data which is then fed into a model that a computer uses to learn what things are supposed to look like. Think of this as the flashcards that we all used when we were in school to learn. The vehicles that we have seen driving around for years from Waymo and Cruise are all collecting data from a wide variety of sensors, which these companies send off for labeling and eventually use in the development of perception algorithms.”
Let’s suppose Tesla was testing LiDAR to incorporate it in new vehicles. We asked Tokic and Salomonsson how much more expensive this sensor would make these $39,990 to $149,490 vehicles be. Tokic gave us his perspective.
“Costs can vary a lot. High-end Velodyne LiDARs can be $10,000s and likely Luminar, etc. Solid-state LiDARs with sufficient range are in the thousands now. All the LiDAR companies are shooting for hundreds and even $100 targets, but that will be years until volumes are sufficient to enable that.”
Salomonsson thinks prices would not be an issue, especially with what is at stake.
“This completely depends on what the automotive company sees as an acceptable safety margin. Using a direct sensing technology such as LiDAR will always result in a better outcome because the distance between objects is known, whereas cameras will always be trying to interpret the scene and are susceptible to interference from ambient light and other environmental conditions. Currently, the entire automotive industry sees direct 3D sensing with LiDAR as a critical part of unlocking safe Level 3+ autonomy and an essential component of many safety-oriented L2 ADAS features. As far as cost, currently, most short-range LiDARs cost around $500, and it’s expected in the next couple of years that number will drop to less than $100.”
That’s the same perception Xpeng has about the technology. The company recently disclosed that the P5 will be its first vehicle with LiDAR. To InsideEVs, Xpeng confirmed it would be more expensive than P5 units without it, but it did not inform the price difference they would have. Xpeng hopes mass-production will allow the LiDARs supplied by Livox to be more affordable.
Volvo is another company that will put LiDAR in its cars. It said vehicles released from 2022 onward will have the equipment, but it did not reply InsideEVs messages until we published this article. Audi has already put LiDAR into production lines.
“The first was the 4 channel Scala Lidar from Valeo that Audi put into their A8 about four years or so ago. All the LiDAR companies have to figure out how to get into ADAS to survive... or go SPAC or acquisition as some have been able to do. The rest will fade earlier and some of these others will just lose. Natural consolidation when you have 80 players for a market that probably can support three to five waiting for AV volumes to become measurable.”
What Tesla was doing with that Model Y with Luminar Hydra LiDARs is something only the company can answer. Considering what it means for Tesla to put these sensors into production cars, the idea that it was just using them for certification is the most plausible one. The excuses around that decision are what deserve second thoughts. If costs were to beat Tesla's concern for safety, its cars would probably fare way worse in crash tests. On the other hand, if Tesla knows LiDARs help redundancy and make trips safer, using them would be a natural choice.