This Tesla Autopilot Limitation Has Drivers Worried: Here's Why

Tesla's advanced driver assistance system continues to make rounds of the news cycle, and not necessarily for good reasons. In addition to lawsuits, the NHTSA initiated another round of inquiries earlier this month, questioning the safety of the tech stack powering Tesla's advanced driver assistance systems (ADAS). However, it's the reliability and the many awareness caveats that keep the tech from reaching its promised potential, as well as a handful of unpredictable risk factors that are neither in Tesla's control, nor the person behind the wheel.

Unpredictability in Tesla's system can stem from something as commonplace as faded lane markings that trick the Autopilot and Full Self-Driving (FSD). Earlier this year, The Washington Post obtained a Tesla engineer's court testimony regarding a fatal crash incident involving a Tesla car that got confused by "faded and nearly obliterated" lane markings on a road — resulting in a crash. It was one of several high-profile incidents in which the Autopilot system was engaged at the time of the incident.

According to The Washington Post, the testimony marked the first time that Tesla revealed a lane-related limitation for its driver assistance tech, revealing the same to the National Transportation Safety Board. In the wake of the crash, Tesla issued a software update that allowed its cars to assess "poor and faded" lane lines more accurately, and even added an alert system that nudged the driver. However, there was no deactivation protocol for Autopilot when the onboard cameras came across faded lanes.

Now, Tesla's corrective update — which seems like a half-baked solution — is not surprising at all, but it does leave the driver in a rather precarious position. Moreover, it's a sticky situation for agencies like the National Highway Traffic Safety Administration, which can force a recall over safety pitfalls, but can not dictate exactly how the carmaker needs to address that safety flaw. Autopilot's failure at handling faded lines on a road is one of them. Should autopilot deactivate the moment it detects unclear lane markings? Should it alert drivers while remaining in control?

Tesla's directives for such a situation are clear, but the execution is not really proactive. Tesla's owner manual notes that if the onboard system fails to accurately determine lane markings, "Autosteer and its associated functions are particularly unlikely to operate as intended." Such situations include "excessively worn" lane markings, when an object is casting shadow over them, lane lines are branching off quickly, or have other visible markings above them.

"The performance of Auto Lane Change depends on the ability of the camera(s) to recognize lane markings," notes the same documents. Tesla also asks drivers to not rely on the automatic lane change systems when external anomalies like rain, fog, or snow could obstruct the cameras, leading to lane misidentification. However, Tesla makes it clear that the individual behind the wheel should be aware of the car's surroundings and retain active control of their car "at all times." 

There are videos on the internet where a Tesla (with Autopilot engaged) can be seen swerving on a road where the lane markings are not perfect visible. "Environmental conditions such as rain and faded lane markings, can affect Autopilot performance," notes another Tesla support document. However, there are a few systemic protocols that can make the lane change system prone to hazards. 

Tesla says when the Auto Lane Change is engaged, but the car faces issues accurately identifying the target lane markings or encounters an obstruction/blind spot, the lane switch process is aborted immediately. Depending on the highway and traffic situation, this could leave the car (and the passengers inside) in a potentially dangerous position. There is another variable here that depends on the car ahead. Tesla's manual mentions that in situations where lane markings are not accurately identified, the system aligns a Tesla with the car ahead of it as an indirect method to identify the driving lane. 

On a high speed route, if the car ahead moves erratically, there could be trouble for the Tesla behind it if Autopilot is engaged. For all anomalous situations where there are issues with lane identification, Tesla says it will flash warning messages on the touchscreen. That said, if the NHTSA's investigations are any indication, drivers may not always have enough time to act, especially when their car is moving at high speed.

There's more to the murky situation here. Back in 2019, researchers demonstrated how adversarial images — which could be something as simple as stickers — could trick a Tesla, steering a Model S towards dangerous traffic. Notably, these stickers placed over a lane marking are perfectly visible to human eyes, but the camera-driven system on a Tesla won't see it at all.

Autopilot's reliability caveats with lane identifications put the onus on users in two ways: First, they need to keep the camera sensors clean. Second, if the camera sensors fail, they must quickly pay heed to the warnings and take control of the car based on their sensory awareness. There's little in terms of a proactive fallback solution here. 

For example, some carmakers rely on a geo-fencing system, and their cars only engage an automated driving system on certain roads — facilitated via GPS. Tesla's support documents don't mention any such GPS-linked system for scenarios where lane markings are tricky to process, and GPS can serve as a secondary route for safe navigation through satellite imagery. 

Tesla, on the other hand, has repeatedly avoided legal heat by claiming that it has put explicit warnings in place about the limitations of Autopilot and Full Self-Driving systems. The automaker isn't technically wrong, as the company markets its ADAS tech at SAE Level 2 autonomy, which strictly specifies that a car's driver must always be ready to wrest control of their vehicle — irrespective of the road ahead and the situation around them.