Get live statistics and analysis of Ted Xiao's profile on X / Twitter

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

After 8 unforgettable years, I have decided to leave Google DeepMind. I feel immensely grateful to have had the opportunity to help transform the dream of general-purpose robot learning from a heretical fringe idea into a normalized technology roadmap. It has been the honor of a lifetime to work on the most challenging and important problems of our time with the brightest, kindest, and most talented colleagues I could have wished for. Thank you to Julian and Vincent for taking a chance on me back in 2017, when a ragtag team at Google Brain began exploring the potential for end-to-end learning on arm farms in the real world. The team has always dreamed big: my “starter project” with Corey and Pierre was to work on a goal-conditioned imitation policy capable of going from any initial condition (latent embedding) to any goal state. That 3-month project turned into a 2-year endeavor! But even though research ambitions were lofty, colleagues and mentors have always been grounded and compassionate by default. Alex H, Karol, Julian, and Sergey supported my vision of concurrent control RL at scale while allowing me the space to grow into a creative researcher on my own terms. The team’s technical progress and my own research taste began to accelerate substantially in 2020, when Kanishka and Karol inspired the whole team to bet big on one single crazy moonshot: a general robot policy that could accomplish thousands of household manipulation tasks. Such an unprecedented group effort was new to the whole team but extremely satisfying—to learn how to harmoniously navigate 0-to-1 real-world systems scaling (robot fleets, teleoperators, scaled learning stacks) alongside rigorous scientific exploration (an objective comparison of the scaling properties of imitation and reinforcement learning). I learned so much from all my comrades-in-arms during this time, and even to this day, many of my research and engineering intuitions draw from the lessons I learned from Eric, Yao, Alex I, Keerthana, and Yevgen. The following period, starting in 2022, was absolutely magical and unique in the breadth and depth of imaginative explorations that I was privileged to contribute to and lead. Exploring the potential of foundation models for robotics changed my research outlook permanently, and projects like SayCan, RT-1, and RT-2 felt like the first magically viral moments when the world started thinking more seriously about what the promise of general and performant embodied AI might look like. When the first generalist VLAs began to reliably perform tasks that we hadn’t collected data for, it was a huge lightbulb moment for our team and the field. During this time, I was immensely inspired by what high agency, manic creativity, and blazing iteration speed can do for research, learning from extremely kind and productive colleagues like Fei, Brian, Andy, Pete, Quan, Harris, and Danny. I applied this approach of wildly creative research to areas I cared about, such as creating better action representations, understanding robot generalization, and leveraging VLMs for data quality and augmentation. I am grateful to teammates who joined me on these adventurous explorations, such as Chelsea, Dorsa, Jonathan, Wenhao, Tianli, Montse, Sean, Austin, Kelly, and Paul. I also deeply appreciate all the academic collaborations during this time—ranging from multi-institution cross-embodiment learning to open-source VLAs to scalable offline evaluation to organizing workshops. Thank you, students, interns, and friends; in particular, Soroush, Jiayuan, Laura, Xuanlin, Kyle, Karl, Oier, Dhruv, Annie, Jensen, Priya, Suneel, Ike, Homanga, Hao, and Xuesu. In the final chapter of my career at GDM, starting in 2024, I became enamored with the science and impact of frontier models and how to harness them properly in robotics. It always fundamentally bugged me that robot learning often looked like “classical” machine learning of just fitting simple distributions with small models, rather than the polished scaled systems and science of how frontier models are developed with pre-training, mid-training, and post-training. I wanted to learn about that world and figure out how to make AGI understand the physical world. I am proud of the progress we have made, and from where we started with Gemini 1.0 to today, the research innovations we have unlocked have placed both Gemini and Gemini Robotics clearly at the forefront of both fundamental world understanding and general VLA control. Thank you so much to my teammates in Embodied Reasoning who make every day bright, interesting, and fun: Fei, Jacky, Laura, Wentao, Annie, Lewis, Ksenia, Mohit, Sean, and Danny. Thank you to friends in Gemini Multimodal who taught me how to frontier model: Xi, Karel, Ishita, and Xudong. Thank you to the VLA whisperers who have shown me how very far innovation and perseverance can take you: Coline, Giulia, Claudio, Alex L, Sumeet, Ashwin, Sudeep, Debi, and Ayzaan. Thank you to mentors throughout the years who have provided shining examples that velocity and impact, and compassion, are not zero-sum: Carolina, Jie, Kanishka, Nicolas, Jonathan, Pierre, Vincent, Karol, Sergey, Chelsea, and Julian. Thank you, thank you, thank you. It has been such an unbelievable adventure, and I am so fortunate to have been part of the crazy team that started the technology breakthroughs transforming the world into one where general and helpful embodied AGI is ubiquitous in society. I will always be #1 GDM fan! As for my own journey, I will be embarking on a new adventure, both familiar and very different, and hope to have more to share soon.

New phenomenon appearing: the latest generation of foundation models often switch to Chinese in the middle of hard CoT thinking traces. Why? AGI labs like OpenAI and Anthropic utilize 3P data labeling services for PhD-level reasoning data for science, math, and coding; for expert labor availability and cost reasons, many of these data providers are based in China. Just as we saw Nigerian/Phillipino cultural influences on dialogue (“delve”), we are starting to see Chinese linguistic influences on reasoning 🤯

Fundamental spatial and temporal understanding is the bedrock upon which robots will learn motor control. These types of Embodied Reasoning capabilities enable policy learning but also inference abilities like image or video conditioning. We just released a new SOTA ER model!

This is hands down the best robotics startup launch I have seen 🔥 Full-stack innovation from hardware design to betting big on UMI to smooth and impressive policies. After 1.5 years of stealth, instantly launching as a top-5 robot foundation model team. Congrats!👏

Gemini 2.5's multimodal embodied reasoning capabilities are making a huge difference for many robotics researchers and industry developers. The Gemini Robotics team has curated a blog post of use cases, recommended prompts, real-world examples, and partner case studies 🧵

Big things come in small and fast packages: announcing the Gemini Robotics On-Device VLA model!🤖 Our new model is optimized for local inference, while showcasing many properties Gemini Robotics excelled at: strong generalization, instruction following, and fast adaptation.

I’ve been building something new at Project Prometheus these past few months. General AI for the physical world is the defining problem of our time, and the founding team is absurdly strong and moving fast. We’re hiring in SF/London/Zurich for real-world ML generalists. DMs open.

Seeing a lot of pessimism about OpenAI on the timeline and I will take a brief break from Gemini shilling to defend OpenAI 🤔 I think many are overly fixating on the fact that recent OpenAI public models are behind the Pareto frontier of cost and capabilities. I don’t think OpenAI necessarily *needs* to be at the bleeding edge of intelligence; what I have really appreciated from OpenAI is their attention to delightful user experiences. Their investments into polished app experiences and UI matter a lot for pretty much all non-API usage; OpenAI did a great job with 4o native image gen and native audio! Shipping consumer facing AI experiences requires not only full-stack infrastructure hyperscaling but also novel research on safety, alignment, and post-training targeting specific user pain points. I think OpenAI is doing well on these fronts, and if they continue to execute on UX, a lot of consumers could continue to find value with these experiences even if future models are 20-30% “worse” than competitors.

Robotics + AI has completely transformed: a perfect storm of AI breakthroughs, hardware innovation, and capital inflow. But are general robot foundation models truly just around the corner? At recent talks, I took an honest look at hype vs. reality to share what’s missing 🧵👇

📢The next milestone for intelligent general-purpose robots has arrived! Announcing Gemini Robotics 1.5, our flagship system which brings breakthroughs from frontier models to the physical world with two new SOTA generalists: the GR 1.5 VLA and GR 1.5 embodied reasoning model 🧵

After 8 unforgettable years, I have decided to leave Google DeepMind. I feel immensely grateful to have had the opportunity to help transform the dream of general-purpose robot learning from a heretical fringe idea into a normalized technology roadmap. It has been the honor of a lifetime to work on the most challenging and important problems of our time with the brightest, kindest, and most talented colleagues I could have wished for. Thank you to Julian and Vincent for taking a chance on me back in 2017, when a ragtag team at Google Brain began exploring the potential for end-to-end learning on arm farms in the real world. The team has always dreamed big: my “starter project” with Corey and Pierre was to work on a goal-conditioned imitation policy capable of going from any initial condition (latent embedding) to any goal state. That 3-month project turned into a 2-year endeavor! But even though research ambitions were lofty, colleagues and mentors have always been grounded and compassionate by default. Alex H, Karol, Julian, and Sergey supported my vision of concurrent control RL at scale while allowing me the space to grow into a creative researcher on my own terms. The team’s technical progress and my own research taste began to accelerate substantially in 2020, when Kanishka and Karol inspired the whole team to bet big on one single crazy moonshot: a general robot policy that could accomplish thousands of household manipulation tasks. Such an unprecedented group effort was new to the whole team but extremely satisfying—to learn how to harmoniously navigate 0-to-1 real-world systems scaling (robot fleets, teleoperators, scaled learning stacks) alongside rigorous scientific exploration (an objective comparison of the scaling properties of imitation and reinforcement learning). I learned so much from all my comrades-in-arms during this time, and even to this day, many of my research and engineering intuitions draw from the lessons I learned from Eric, Yao, Alex I, Keerthana, and Yevgen. The following period, starting in 2022, was absolutely magical and unique in the breadth and depth of imaginative explorations that I was privileged to contribute to and lead. Exploring the potential of foundation models for robotics changed my research outlook permanently, and projects like SayCan, RT-1, and RT-2 felt like the first magically viral moments when the world started thinking more seriously about what the promise of general and performant embodied AI might look like. When the first generalist VLAs began to reliably perform tasks that we hadn’t collected data for, it was a huge lightbulb moment for our team and the field. During this time, I was immensely inspired by what high agency, manic creativity, and blazing iteration speed can do for research, learning from extremely kind and productive colleagues like Fei, Brian, Andy, Pete, Quan, Harris, and Danny. I applied this approach of wildly creative research to areas I cared about, such as creating better action representations, understanding robot generalization, and leveraging VLMs for data quality and augmentation. I am grateful to teammates who joined me on these adventurous explorations, such as Chelsea, Dorsa, Jonathan, Wenhao, Tianli, Montse, Sean, Austin, Kelly, and Paul. I also deeply appreciate all the academic collaborations during this time—ranging from multi-institution cross-embodiment learning to open-source VLAs to scalable offline evaluation to organizing workshops. Thank you, students, interns, and friends; in particular, Soroush, Jiayuan, Laura, Xuanlin, Kyle, Karl, Oier, Dhruv, Annie, Jensen, Priya, Suneel, Ike, Homanga, Hao, and Xuesu. In the final chapter of my career at GDM, starting in 2024, I became enamored with the science and impact of frontier models and how to harness them properly in robotics. It always fundamentally bugged me that robot learning often looked like “classical” machine learning of just fitting simple distributions with small models, rather than the polished scaled systems and science of how frontier models are developed with pre-training, mid-training, and post-training. I wanted to learn about that world and figure out how to make AGI understand the physical world. I am proud of the progress we have made, and from where we started with Gemini 1.0 to today, the research innovations we have unlocked have placed both Gemini and Gemini Robotics clearly at the forefront of both fundamental world understanding and general VLA control. Thank you so much to my teammates in Embodied Reasoning who make every day bright, interesting, and fun: Fei, Jacky, Laura, Wentao, Annie, Lewis, Ksenia, Mohit, Sean, and Danny. Thank you to friends in Gemini Multimodal who taught me how to frontier model: Xi, Karel, Ishita, and Xudong. Thank you to the VLA whisperers who have shown me how very far innovation and perseverance can take you: Coline, Giulia, Claudio, Alex L, Sumeet, Ashwin, Sudeep, Debi, and Ayzaan. Thank you to mentors throughout the years who have provided shining examples that velocity and impact, and compassion, are not zero-sum: Carolina, Jie, Kanishka, Nicolas, Jonathan, Pierre, Vincent, Karol, Sergey, Chelsea, and Julian. Thank you, thank you, thank you. It has been such an unbelievable adventure, and I am so fortunate to have been part of the crazy team that started the technology breakthroughs transforming the world into one where general and helpful embodied AGI is ubiquitous in society. I will always be #1 GDM fan! As for my own journey, I will be embarking on a new adventure, both familiar and very different, and hope to have more to share soon.

📢 First contact between a frontier model and robots! Gemini Robotics is a SOTA generalist Vision-Language-Action model bringing frontier model intelligence to the physical world. It's an extremely capable model enabling dexterous, steerable, and general robot control. 🧵⬇️

Seeing a lot of pessimism about OpenAI on the timeline and I will take a brief break from Gemini shilling to defend OpenAI 🤔 I think many are overly fixating on the fact that recent OpenAI public models are behind the Pareto frontier of cost and capabilities. I don’t think OpenAI necessarily *needs* to be at the bleeding edge of intelligence; what I have really appreciated from OpenAI is their attention to delightful user experiences. Their investments into polished app experiences and UI matter a lot for pretty much all non-API usage; OpenAI did a great job with 4o native image gen and native audio! Shipping consumer facing AI experiences requires not only full-stack infrastructure hyperscaling but also novel research on safety, alignment, and post-training targeting specific user pain points. I think OpenAI is doing well on these fronts, and if they continue to execute on UX, a lot of consumers could continue to find value with these experiences even if future models are 20-30% “worse” than competitors.

New phenomenon appearing: the latest generation of foundation models often switch to Chinese in the middle of hard CoT thinking traces. Why? AGI labs like OpenAI and Anthropic utilize 3P data labeling services for PhD-level reasoning data for science, math, and coding; for expert labor availability and cost reasons, many of these data providers are based in China. Just as we saw Nigerian/Phillipino cultural influences on dialogue (“delve”), we are starting to see Chinese linguistic influences on reasoning 🤯

It is a travesty that all AGI labs are sprinting away from the physical world and towards digital companions, customer service bots, enterprise software, and coding agents. That the real world is difficult and messy and unforgiving is exactly why we need to be prioritizing it.

Vibe check: Any behavior which a human can teleoperate a robot to do, an end-to-end policy can be learned. That is, for *any* initial condition which a human can successfully teleoperate from, an automated policy can match in performance/robustness.

humanoid is robot vacuum is robot car is robot space rocket is robot lab is robot warehouse is robot factory is robot at the end of the day, everything is robot

Visiting Zurich🇨🇭 this week! Would love to learn what researchers in AI and robotics are excited about. Who should I chat with?

I’ve been building something new at Project Prometheus these past few months. General AI for the physical world is the defining problem of our time, and the founding team is absurdly strong and moving fast. We’re hiring in SF/London/Zurich for real-world ML generalists. DMs open.

live look at the robotics + AI landscape

> year is 2031 > be me, token farmer in the slop mines > bring in 20k novel tokens for weighing at the Token Utility station > my token haul is two stddevs over the compressibility threshold, phew > get my daily blueprint nutrition allocation > trickle down tokenomics utopia

I’ve gradually come around to two paths to embodied AGI that I was very skeptical of before: 1️⃣ solving robotics via reasoning 2️⃣ solving robotics via world modeling I was previously doubtful not of these approaches themselves, but of timelines and efficiency; the “optimal” algorithms or approaches to use may not be the ones which make the most progress. Hardware and software lotteries are very real, especially in robotics where prohibitive CapEx and OpEx limit the search space of research ideas. So when trillions of dollars and millennia of R&D hours of effective investment have been poured into pre-training, scaling, and inference optimization for internet-scale LLMs, this is a powerful foundation to build off, and a very tough baseline ecosystem to beat — even if suboptimal for something like robotics. But recent progress and momentum in reasoning and generative world modeling give me hope that we’ll see alternative approaches to unlocking general-purpose robotics, with competitive pace of progress! More entropy is better when we are still so early. Two directions I would love to see more signals on in 2025: 1️⃣ For reasoning / search-based models, to show that inference-time compute scaling can be amortized and distilled into training. While minutes (hours?!) of thinking processes might make sense in return for very strong results in coding or math, this is simply a non-starter in real world robotics. Additionally, I would love to see evidence of how well “thinking harder” improves physical embodied reasoning performance. The types of intelligence required for low-level control are very different than the types of intelligence required for solving math Olympiad problems or debugging large software codebases. 2️⃣ For generative world modeling, to show that the important physical properties of the world are modeled in a non-trivial manner. Beyond basic concepts like object permanence or the forward arrow of time, how well do approaches like action-conditioned video prediction (which are generally optimized for *aesthetic reconstruction*, rather than physical grounding) capture the details of what’s important for robotic control? I would like to see how world models capture different behavior modes, capture different levels of optimality (model not only successful world interactions, but also failures), and different robot embodiments (assumptions valid in for a factory robot may be very bad for modeling home robots). The future is bright! For both 1️⃣ and 2️⃣, progress is orders of magnitudes what we saw in 2023. Looking forward to an even more diverse and interesting 2025 ⭐️