Computer Science Thesis Proposal

Location:
In Person - McWilliams Classroom, Gates Hillman 4303

Speaker:
YUE (SOPHIE) GUO , Ph.D. Student, Computer Science Department, Carnegie Mellon University
http://sophieyueguo.com/

Action Advising in Reinforcement Learning: Enhancing Teacher-Student Policy Transfer

As human students benefit from teachers’ advice to accelerate their learning, so could agents benefit from advice. Agents can not only learn from humans, e.g. via human expert demonstrations, but also from other agents. This thesis proposal focuses on action advising, a knowledge transfer technique built upon the teacher-student paradigm of reinforcement learning. In this approach, the teacher agent provides action advice calculated from its policy given the student’s observations. Although action advising has been studied over the past decade, the focus of the related work has primarily been on when to advise. In this proposal, we extend the current state-of-the-art by studying the following additional challenges: 

  1. In existing work, advice is given without explaining the rationale behind it; 
  2. In many situations, the teacher might be suboptimal in a new environment, but there are no current approaches that enable the student to discern when some particular pieces of advice might not be applicable; 
  3. No present techniques enable the teacher to evaluate the quality of its advice before giving it to the student; 
  4. The teacher with a fixed pre-trained policy might not be able to provide flexible advice.

In this thesis proposal, we present our solutions to tackle the aforementioned challenges and exhibit the empirical effectiveness of our proposed methods. We also propose potential pathways for subsequent research, serving as a guide to complete this thesis. Our ultimate goal is to delve into the intricacies and potentials of action advising in reinforcement learning, thereby directing future advancements in the field.

Thesis Committee:
Katia Sycara (Chair)
Fei Fang
Zico Kolter
Matthew E. Taylor (University of Alberta)

Add event to Google
Add event to iCal