5th Year Master's Thesis Presentation - Allen Zheng
April 24, 2026 4:30PM—6:00PM
Location:
In Person
-
Newell-Simon 4305
Speaker:
ALLEN ZHENG,
Master's Student, Computer Science Department. Carnegie Mellon University
We introduce a framework that incorporates attention over past states into reinforcement learning (RL) in two complementary ways. First, we use an attention mechanism over a trajectory buffer of previously visited states to construct a history-aware critic, replacing the standard neural network critic with an estimate computed as an attention-weighted average over stored values. Second, we replace GAE, which is computed through a single trajectory, with an attention-weighted advantage that, in addition to stepping forward through time, also steps according to similar states. States more similar to the current one contribute more to the return estimate, providing a smooth, similarity-weighted alternative to the sequential rollout. Together, these two mechanisms reduce variance by pooling signal across similar states rather than relying on a single trajectory.
Thesis Committee
Geoff Gordon (Chair)
Jeff Schneider
Additional Information
For More Information:
amalloy@cs.cmu.edu