A robot which can pour its master another drink has been created by scientists.
Researchers at Cornell’s Personal Robotics Lab have taught their creation to forsee human action with the task of refilling a person’s cup when it was nearly empty, without having to be asked – great for the silent and legless types at parties.
However, there are pitfuls. To be able to carry out the task the robot has to plan its movements in advance and then follow the plan. But if a human sitting at the table happens to raise the cup and drink from it, the robot might pour a drink into a cup that isn’t there.
Nevertheless there’s also benefits. In another test, the robot observed a human carrying an object toward a refrigerator and helpfully opened the refrigerator door.
From a database of 120 3-D videos of people performing common household activities, the robot has been trained to identify human activities by tracking the movements of the body – reduced to a symbolic skeleton for easy calculation – breaking them down into sub-activities like reaching, carrying, pouring or drinking, and to associate the activities with objects. Since each person performs tasks a little differently, the robot can build a model that is general enough to match new events.
Observing a new scene with its Microsoft Kinnect 3-D camera, the robot identifies the activities it sees, considers what uses are possible with the objects in the scene and how those uses fit with the activities.
It then generates a set of possible continuations into the future – such as eating, drinking, cleaning, putting away – and finally chooses the most probable. As the action continues, it constantly updates and refines its predictions.
The research was supported by the U.S. Army Research Office, the Alfred E. Sloan Foundation and Microsoft.
Hema S. Koppula, Cornell graduate student in computer science, and Ashutosh Saxena, assistant professor of computer science, will describe their work at International Conference of Machine Learning, June 18-21 in Atlanta, and the Robotics: Science and Systems conference June 24-28 in Berlin, Germany.
