Researchers from Google’s Robotics team recently announced PaLM-E, a combination of their PaLM and Vision Transformer (ViT) models designed for controlling robots. PaLM-E handles multimodal input data from robotic sensor and outputs text commands to control the robot’s actuators. Besides performing well on several robotics tasks, PaLM-E also outperforms other models on the OK-VQA benchmark.
This is a companion discussion for the article “Google's PaLM-E Combines Vision and Language AI for Robot Control - https://www.infoq.com/news/2023/06/google-palm-e-robot/” submitted on the community's news feed.
Reply to this topic to share your thoughts on this article.