Tesla Shows Optimus Teslabots in Factory Demo

by

Tesla showed new video of improved Optimus Teslabots and some are doing some useful work in factories. The hands have been improved and hands and AI are the keys to successful humanoid bots. Tesla Engineer Milan Kovac reports that Tesla has deployed a couple bots at one of the Tesla factories.

SanctuaryAI and Tesla have the best robotic hands. SanctuaryAI have the full copy of the hand without removing knuckles. Tesla did copy SanctuaryAI but both are copying the human hand.

Elon says Tesla’s Optimus humanoid robot will have a new hand later this year that has 22 degrees of freedom. The Optimus’ hand had 11 degrees of freedom this past December.

Teslabot Engineer Milan Kovac Gives an Update

Over the past couple of months, our awesome manufacturing team has built more bots for us to work on, and collect AI data from!

We’ve trained and deployed a neural net allowing Optimus to start doing useful tasks, such as picking up battery cells coming down a conveyor and precisely inserting them into a tray.

This neural net is running entirely end-to-end, meaning that it only consumes video coming from the bot’s 2D cameras, as well as on-board proprioceptive sensors, and produces joints control sequences directly. It runs entirely on the bot’s embedded FSD computer, powered by the on-board battery. It is designed such that a single neural net can perform multiple tasks as we add more diverse data to the training process.

While not being perfect yet and still a little slow, we’re seeing increasingly high success rates with less frequent misses. We’re also training Optimus to recover from failure cases, and are seeing spontaneous corrections happen.

We’ve deployed a couple bots at one of our factories, where they’re being tested daily at the real workstations and continuously improving!

Optimus also now regularly takes long walks across the office without falling 🙂

Further work is on-going to make it move faster, as well as dealing with more adverse terrains – all without sacrificing the human-like nature of it. We’re also focusing on repeatability across the fleet, training the neural net to deal with dynamic calibration & small bot-to-bot variance.

Jim Fan- Ai Expert Analysis on Teslabot

Congrats to
@Tesla_Optimus team on another stellar update! The video gives us a peek at their human data collection farm, which I believe is Optimus’ biggest lead. What does it take to build such a pipeline?

Optimus nailed all of the following:

1. Optimus hands are among the best 5-finger, dexterous robot hands in the world. It’s got tactile sensing, 11 degrees of freedom (DOF) compared to many competitors with only 6-7 DOF, and robustness to withstand lots of object interactions without constant maintenance.

2. Teleoperation software: we can see that the human operators are wearing VR goggles and gloves. It is very non-trivial to set up the software to have first-person video streamed in and precise control streamed out, while maintaining extremely low latency. Humans are highly sensitive to even the smallest delay between their own motions and the robot’s. Optimus has a fluid whole body controller that enacts the human poses in real-time.

3. Sizeable fleet: you need more than one robot to collect data in parallel, well-trained human contractors taking multiple shifts per day (preferably 24/7), and an on-call maintenance crew to make sure that the robots are always busy. That’s a ton of operational complexity that academic research labs don’t even think of.

4. Tasks & environments: it’s equally important to figure out *what* to teleoperate. Currently, most such efforts are demo-driven: collect data on the tasks that you want to put into a social media video. But solving general-purpose robots requires us to think carefully about the distribution of tasks and environments. From 43″-51″ in the video, we can see factory & household settings like moving batteries, handling laundry, sorting daily objects into shelves.

It’s an open-ended research question: if you only have the budget to collect training data for 1,000 tasks, what would you pick to maximize skill transfer and generalization?

Closing thought: teleoperation is a necessary but insufficient condition to solve humanoid robotics. It fundamentally does not scale. More about this later.

4 thoughts on “Tesla Shows Optimus Teslabots in Factory Demo”

  1. Tesla’s Optimus is more impressive than their FSD, considering they are working on it for only a few years compared to FSD. Robots will replace humans in factories before they replace human drivers. Lining up objects is very linear compared to the extremely non-linear situations that can occur in the road.

  2. Wake me when a Tesla bot can mine a coal face, rough neck on a drill rig. Assemble cars at a Tesla factory, repair a deep sea telecom cable, milk the cows, feed the chickens, harvest lettuce, spinach, melons, oranges, and grapes.

    • There are 2 types of jobs on that list: things I can’t do either and things that can be automated without a humanoid or a terribly advanced level of AI. 😂

Comments are closed.