The small robot, “Misty,” sits on the tabletop and watches Isabelle Boucher as she mixes batter to make cookies.
“An ingredient is still missing,” Misty helpfully points out in a female automaton voice.
The Franklin & Marshall College junior glances around as she stirs the bowl, then looks directly at the robot and says, “I don’t know what’s missing?”
“You still need to add vanilla to your recipe,” the robot says, and turns its head and nods at the bottle of vanilla. “The missing vanilla is right over there by the flour.”
The video shows the research that Boucher and senior Nhi Phan have been doing to program the social robot to offer assistance to human tasks.
“I learned so much this summer about not just artificial intelligence, but also how different technological components work together and the importance of that cohesiveness,” Phan says.
Assistant Professor of Computer Science Willie Wilson has been working with the two students on the research this year, which continues into the fall.
“We will be working on improving the robot's ability to recognize when a user needs help, improving upon how it processes the user's language, extending the types of eye-gaze patterns that are recognized, and integrating a model of multiple levels of need,” he says.
Eye gazes and questions prompted Misty to assist by pointing out and talking, not by physical interaction.
“It doesn’t provide physical assistance,” Wilson says. “The idea of a socially assistive robot is to provide conversational, verbal and sometimes environmental cues.”
The robot is not programmed to dominate, but to assist the human in tasks.
“In designing the robot, we incorporate concepts from psychology, philosophy and occupational therapy,” Wilson says. “One concept in each of these areas is the notion of human autonomy, and we design our robots to protect and maintain the autonomy of the user.”
That aspect of the research intrigued Phan.
“It was awesome to work with a socially assistive robot while it was still at its beginning stages and kind of leaving my own personal mark on the research,” she says. “To think about how various academic disciplines were related to our research and seeing my ideas and thoughts unravel into a physical representation was also really cool!”
Phan says the research also provided her practical experience.
“It not only taught me more in terms of technical things, but it also introduced me to collaborative projects as well as timelines,” she said.