Computer Science Masters Thesis Presentation
— 11:30am
Location:
Virtual Presentation
-
Remote Access - Zoom
Speaker:
YUFEI WANG
,
Masters Student
https://yufeiwang63.github.io/
Towards Broader and More Efficient Object Manipulation via Deep Reinforcement Learning
Reinforcement learning (``RL'') has achieved great success in many robotic object manipulation tasks, such as pushing, grasping, tossing, inserting, and more. However, there remain some challenges in applying RL to a broader range of object manipulation tasks in the real world. First, it is challenging to obtain a reward function directly from high-dimensional images in the real world for applying RL to do object manipulation. Second, although great progress has been made in the regime of rigid object manipulation, manipulating deformable objects remains challenging due to its high dimensional state representation, and complex dynamics. In this thesis, we aim to push forward the application of deep RL to object manipulation, by proposing a novel algorithm that learns a self-supervised reward function directly from images that achieves better sample-efficiency, and by creating a suite of benchmarking tasks for evaluating deep RL for deformable object manipulation.
Specifically, for obtaining a reward function directly from images, current image-based RL algorithms typically operate on the whole image without performing object-level reasoning. This leads to inefficient goal sampling and ineffective reward functions. In this thesis, we improve upon previous visual self-supervised RL by incorporating object-level reasoning and occlusion reasoning. We use unknown object segmentation to ignore distractors in the scene for better reward computation and goal generation; we further enable occlusion reasoning by employing a novel auxiliary loss and training scheme. We demonstrate that our proposed algorithm, ROLL (Reinforcement learning with Object Level Learning), learns dramatically faster and achieves better final performance compared with previous methods in several simulated visual control tasks.
To facilitate the research of using deep RL to explore the challenges of deformable object manipulation, in this thesis, we present SoftGym, a set of open-source simulated benchmarks for manipulating deformable objects, with a standard OpenAI Gym API and a Python interface for creating new environments. Our benchmark will enable reproducible research in this important area. Further, we evaluate a variety of algorithms on these tasks and highlight challenges for reinforcement learning algorithms, including dealing with a state representation that has a high intrinsic dimensionality and is partially observable. The experiments and analysis indicate the strengths and limitations of existing methods in the context of deformable object manipulation that can help point the way forward for future methods development.
Thesis Committee:
David Held (Chair)
Katerina Fragkiadaki
Deepak Pathak
Zoom Participation. See announcement.
For More Information:
amalloy@cs.cmu.edu