Vittorio Perera

Language-Based Bidirectional Human And Robot Interaction Learning For Mobile Service Robot Degree Type: Ph.D. in Computer Science
Advisor(s): Manuela Veloso
Graduated: August 2018

Abstract:

We believe that it is essential for robots that coexist with humans to be able to interact with their users seamlessly. This thesis advocates the use of language as a rich and natural interface for the interaction between robots and humans. We assume that a mobile service robot, such as the CoBot robot, is equipped with domain information about its environment and is able to perform tasks involving autonomous navigation to desired goal positions. The thesis provides the robot and the human with the ability to interact in natural language, introducing a novel bidirectional approach for the exchange of commands and information between a robot and its users.

In the human-to-robot direction of interaction, we assume that users provide a high-level specification of what the robot should do. This thesis enables a mobile service robot to understand (1) requests to perform tasks, and (2) questions about the robot experience as stored in its log files. Our approach introduces a dialogue-based learning of groundings of natural language expressions to robot actions and operations. These groundings are learned into knowledge bases that the robot can access.

In the robot-to-human interaction direction, this thesis enables a robot to match the detail of the explanations it provides to the user's request. Moreover, we introduce an approach that enables a robot to pro-actively report, in language, on the outcome of a task after executing it. The robot contextualizes information about the task execution by comparing it with its past experience.

In a nutshell, this thesis contributes a novel, language-based, bidirectional interaction approach for mobile service robots, where robots learn to understand and execute commands and queries from users, and take the initiative to offer information, in language, to users about their experience. So, the language exchange can be initiated by the robots, as well as by the humans.

We evaluate the work both on the actual CoBot robots, and on constructed simulated and crowd-sourced data.

Thesis Committee:
Manuela Veloso(Chair)
Jaime Carbonell
Stephanie Rosenthal
Xiaoping Chen (University of Science and Technology of China)

Srinivasan Seshan, Head, Computer Science Department
Andrew W. Moore, Dean, School of Computer Science

Keywords:
Human-robot interaction, natural language processing, spoken language, dialogue