Toyota Research Institute

TRI welcomes our 2024 summer #interns to our Los Altos, CA, and Cambridge, MA, offices! We are excited to see what new ideas and inspiration they will bring and take away from their experiences here. 

Wishing everyone a very happy and safe Fourth of July! 🎉

Gill Pratt, Chief Scientist for Toyota Motor Corporation and TRI CEO, addressed a group of energy stakeholders in Washington, D.C., this week at The Energy Future Forum. He discussed Toyota’s plans to reduce as much carbon as soon as possible through Toyota’s multi-pathway approach. Did you know that we can save about 45% more C02 from being emitted into the atmosphere by manufacturing 90 HEVs vs 1 BEV with the same amount of lithium? We call this Carbon Return on Investment (CROI). Listen how:

Meet José Saavedra-Cabrera, a People Business Partner at TRI! In this Spotlight, José shares his motivations for fostering a positive work environment where everyone can feel comfortable bringing their full, authentic selves. Come see how he celebrates #Pride Month 🏳️🌈, his most influential LGBTQIA+ figures, and his favorite spots in SF!

Thanks for joining us at #CVPR! Check out some of our featured papers and highlights from our networking event. Understanding Video Transformers via Universal Concept Discovery. – https://lnkd.in/g6xd2W8P pix2gestalt: Amodal Segmentation by Synthesizing Wholes – https://lnkd.in/gaYsS-Pe Towards Realistic Scene Generation with LiDAR Diffusion Models –https://lnkd.in/gx4ZAC9K See you all next year!

“Once language-based representations are learned, their semantics can be used to share autonomy between humans and vehicles or robots, promoting usability and teaming,” said Guy Rosman, Senior Research Scientist at TRI and manager of the Human Aware Interaction and Learning team. UVA Engineering and TRI are collaborating on research to enable #AI to understand the meaning of drivers’ actions, learn more: https://lnkd.in/gsgeYBdZ

Researchers from TRI, Stanford, UC Berkeley, Google Deepmind, and other labs have introduced OpenVLA, an open-source vision-language-action (VLA) model trained on a diverse collection of real-world robot demonstrations. Foundation models are becoming vital to advancing #robotics, but adoption has been limited by its closed nature and the lack of best practices. OpenVLA aims to make these models more accessible and customizable to futher accelerate robotics research. Learn more in this VentureBeat article: https://lnkd.in/gM79y6EZ

Happy Juneteenth from TRI! Our African Diaspora and Allies ERG (Employee Resource Group) led celebrations to commemorate the ending of slavery in the US. At TRI, we recognize that many of the technological and social advances we all experience are shaped by the talents and ingenuity of Black communities - despite their ongoing struggle and inequity. We are proud to foster an environment where team members are encouraged to be allies and stand up when we see others not acting from a place of kindness, respect, and best intentions.

Stanford, Berkeley, and TRI, along with collaborators from Google, MIT, and Physical Intelligence, have just released OpenVLA: a fully open-source (code, weights, and data) 7B Vision-Language-Action behavior model for robotics. Similar in approach to Google's RT-2-X, OpenVLA is trained on the Open X-Embodiment and DROID datasets, predicts actions directly from robot sensor input, and is deployable on several common manipulation platforms. I'm excited to see this work (and future work in the same vein) accelerate robotics research in the same way models pretrained on ImageNet led to rapid development in computer vision. By providing a base model to explore, experiment with, probe for limitations, iterate upon, and adapt to downstream tasks, OpenVLA will be a useful tool for further research. A few takeaways from this work: 1. Vision-Language Models (VLMs), even when trained without action data, are surprisingly effective base models for learning single skills. 2. Strong single-skill approaches (like DiffusionPolicy or ACT) perform very well within the distribution of their training data. If your test conditions are IID relative to your training data, fitting that data with an expressive model works well. The true promise of pretrained base models lies in robustness, generalization, and enabling graceful failure, as seen in other areas of machine learning. 3. Inference speed remains quite reasonable with 7B models. OpenVLA doesn't require cloud resources to deploy – it can be run locally on consumer GPUs at multiple inference cycles per second, even without optimization techniques like compilation or speculative decoding. There is still considerable potential for further improvement. Congratulations to lead authors Moo Jin Kim, Karl Pertsch, and Siddharth Karamcheti and other collaborators Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn.

Last month, TRI held its first in-person company all-hands since the pandemic! It has been over eight years since we started our mission to create new tools and capabilities focused on improving the quality of life for individuals and society. A huge thanks to all of our team members across every function for powering our drive toward exciting breakthroughs and innovations! If you want to join us on this journey, please find all our open roles here: https://lnkd.in/d5Mtz5_R