5th Year Thesis Presentation - Lawrence Chen

— 2:30pm

Location:
In Person and Virtual - ET - Traffic21 Classroom, Gates Hillman 6501 and Zoom

Speaker:
LAWRENCE CHEN, Master's Student, Computer Science Department, Carnegie Mellon University
https://lawrencedchen.com/

Spline-FRIDA: Enhancing Robot Painting with Human Brushstroke

A painting is more than just a picture on a wall; a painting is a process comprised of many intentional brush strokes, leading to a performance far richer than the final output. The shapes of individual strokes are an important component of a painting's style. This is especially true for sparse sketches, where individual strokes are likely to be visible. Prior work in modeling brush stroke trajectories either does not work with real-world robotics or is not flexible enough to capture the complexity of human-made brush strokes. In this work, we aim to develop a robotic drawing agent with controllable stroke-level style based on human trajectories. 

To achieve this, we develop a framework to collect brush trajectories from human artists on a real canvas. We model these trajectories with an autoencoder. Finally, we incorporate the autoencoder into the planning pipeline in the FRIDA robotic painting system. We find that, off-the-shelf, FRIDA's brush stroke renderer struggles or fails to learn the complex trajectories from the human demonstration data, especially with narrow brushes or markers. We present a novel brush stroke renderer that is capable of generalizing to complex, human-made brush strokes while maintaining a small Sim2Real gap. Our code is open sourced along with the dataset of drawing trajectories collected from people using real-world drawing tools. 

Thesis Committee: 

Jean Oh (Chair)
Jim McCann 

Additional Information