Tesla AI and Humanoid Robot Leads Describe AI at Tesla

Tesla head of AI describes the past decade of AI at Tesla. Ashok Elluswamy @aelluswamy clarified that this was his personal note that I decided to write to explain Elon’s importance to Tesla. While the timing is partly in light of the shareholder vote, I truly believe everything I wrote.

@elonmusk has been the key driver of AI and autonomy at Tesla. He has always pushed us to achieve great things, even when such ideas were seemingly impossible at the time. Some examples:

* Back in 2014, Autopilot started on a ridiculously tiny computer that only had ~384 KB of memory and puny compute (didn’t even have native floating point arithmetic). He asked the engineering team to implement lane keeping, lane changing, longitudinal control for vehicles, curvature, etc. Many, even in the team, thought that the request was crazy. Nonetheless, he never gave up and pushed the team to achieve this very difficult goal. In 2015, beyond all odds, Tesla shipped the world’s first Autopilot system. The second closest such product only came to market many years later.

* In 2016, Tesla started doing all of the computer vision required for Autopilot in-house instead of depending on external vendors. Many people thought it was insane to bet the product on developing the vision system from scratch within a few months, which had taken other companies a decade or more. Yet, we achieved this target within eleven months. This was a strategically important move that started the development of a strong AI team at Tesla.

* Not only did he push for strong AI software, but also for powerful AI hardware. Tesla, which others thought was just a car company, was making custom silicon to run neural networks efficiently. This hardware that was originally designed in 2017, came to production in February 2019 and remains extremely competitive with hardware coming out to date. For reference, this five year old AI computer has roughly 8x the AI inference compute as the state-of-the-art Apple M3 chip. It is still able to run the latest end-to-end neural networks built on top of the latest AI technology.

* He was the one who bet on vision and AI to solve autonomy instead of relying on sensor crutches and high-definition maps. For anyone who has experienced the latest versions of FSD, it might be obvious that it can see all the important things and drive the car based on pure vision. However, back in 2020 and earlier it wasn’t obvious to most. In fact, many “experts” in the field ridiculed Tesla and Elon for these choices. We have proved them wrong by shipping supervised FSD to millions of cars and shown that with good AI software, the car is able to handle the complexities of city driving such as making turns, handling intersection, yielding to pedestrians etc., just by seeing outside. In fact, we even removed the radars and ultrasonics to just really focus on the heart of the problem, which is AI. Today, it’s almost paradoxical that, Teslas have the least amount of raw sensors, yet have the most autonomous capability compared to any production car. Pulling off such a contrary bet was only possible because of his extreme conviction and deep understanding of this problem.

* He kickstarted the work on humanoid robots at Tesla in 2021, again before any ChatGPT or other obvious examples of the rise of AI. Just like the vehicle autonomy, Optimus is also being developed to be competent, scalable, and cost-effective in order to widely serve the world.

* I [Ashpk] could go on, but plainly, Elon is critical for Tesla’s success in AI. It is his combination of deep technical understanding, insane perseverance and relentless hard work that have positioned Tesla to be a leader in real-world AI. Elon’s technical intuition to make these important decisions way before others see it is unmatched. If not for Elon’s ambition, Tesla might have dwindled to become just another car company. In the future, fully autonomous cars and useful household robots will be common place and the world will think that this was how it was always supposed to be. Until then, we need Elon Musk to push the frontier, because he sees it already.

https://t.co/yUqvdS7JOf

— Ashok Elluswamy (@aelluswamy) June 9, 2024

@aelluswamy summarized the past decade very well.

When I hopped in my friend’s model S in 2015, it hit me in a weird way: that car was running software all around, had a nice big touch screen (finally, an actual “screen”), was receiving frequent feature updates over the air, and… https://t.co/d6NmcELQ2W

— Milan Kovac (@_milankovac_) June 9, 2024

@aelluswamy summarized the past decade very well.

When I [Milan Kovac, head of Teslabot] hopped in my friend’s model S in 2015, it hit me in a weird way: that car was running software all around, had a nice big touch screen (finally, an actual “screen”), was receiving frequent feature updates over the air, and came with a mobile app for essential remote controls. How could a big car company build something like this? Didn’t make sense back then (and still today for many).

My interview with the Autopilot team early 2016 was very different than most interviews I’d ever taken. We first had a technical discussion on something I had built before, with several engineers and the executive in charge of the group at the time. An actual white board chat where we were all bouncing ideas on the matter together. The only places I’d ever seen the face of anyone above, at best, a team lead during interviews was at some small startups.

Every 1:1 interview that followed was similarly practical. Real coding situations you’d encounter as an engineer, not useless LeetCode trick questions typically found in other big companies’ interviews. When we got done, the recruiter walked me through the office. Everyone was sitting literally next to each other: autopilot software, hardware, vehicle firmware, and many other teams interacting live without friction. Eventually, we walked past @elonmusk
‘s desk and he was sitting right there, next to the engineering teams. Not in any separate ivory tower.

One week in the job, and I was already in a team meeting with him brainstorming Autopilot technical challenges, exactly how it went during my interview. And that went on almost every single week, for the 8+ years that followed.

It soon became pretty clear that Elon was directly behind that culture of pragmatic innovation, percolating through all aspects of the company.

Week after week, I’ve witnessed that relentless drive to build features that make people’s lives better and safer, removing roadblocks and unnecessary layers one after another, systematically drilling down to the fundamental “why” – all of this while sleeping at the factory during Model 3 production hell, designing new vehicles, working on BOM reductions, and launching new factories across the globe. During that entire time, through all these chapters, news headlines and other difficult company-wide moments, and while landing rockets on drone ships in the ocean, Elon was still sitting with us in a room every week, often more, with the only objective of building things that will change humanity for the better.

When he announced Tesla would soon start a humanoid robotics program to fuel a future of abundance at AI Day 2021, many once again laughed and doubted. Two years into the program, and Tesla is actively testing early versions of what could well be the first full-fledged humanoid robots equipped with articulated hands autonomously conducting real tasks in a real factory via an end-to-end neural net, running entirely on the bot’s compute hardware. And again, using 2D cameras only.

Whether at Tesla or not, I’d say the same: without Elon, none of any of these amazing things would have ever happened. I can only imagine what a lesser future we’d be living without his involvement and dedication.