Google has released a new open source benchmark that it says will allow robots to perform tasks by writing their own code in response to instructions written by humans.
The company has launched a new website (opens in a new tab) to reveal the "Code as Policies" (CAP), in which prompts written in plain English can be interpreted in programs generated by a language model (LMP). ) written in Python code.
CAP is the successor to PaLM-SayCan (opens in a new tab), a project that similarly allowed a physical assistant robot to be commanded through plain language commands. CAP promises to allow more complex tasks to be performed with greater precision, in part because it allows machines to write their own code.
Autocoding robots
In a blog post (opens in a new tab) about the launch of CAP, Google Research intern Jacky Liang and research scientist Andy Zeng describe the motivation for the technology and what it could mean for the future.
“What if, by receiving instructions from people, robots could autonomously write their own code to interact with the world? Given the natural language instructions, current language models are very good at writing not just generic code but, as we have discovered, code that can also control the actions of robots.
But now might not be the time to throw away your laptop for programming. In tests, Google researchers demonstrated simple commands with a similar structure. The test robots were able to "draw a 5cm hexagon around the middle" and "place the blocks in a horizontal line near the top".
In the companion paper (opens in a new tab), titled "Code as Policies: Language Model Programs for Embedded Control," the project team admits that CAP currently lacks the ability to process commands that are particularly abstract or complex or perceive trajectory descriptions. The team's approach also failed to take into account the impossible commands transmitted via CAP.
In theory, the open source nature of Google's "bot-centric" implementation of LMP in Python could result in solutions to these problems being implemented much faster. The CAP website also has builds (opens in a new tab) via Github and an interactive demo (opens in a new tab) via Google Colab to describe how robots "write" code in response to requests. commands.