Artificial Intelligence Seminar - Keegan Harris

— 1:00pm

Location:
In Person and Virtual - ET - ASA Conference Room, Gates Hillman 6115 and Zoom

Speaker:
KEEGAN HARRIS, Ph.D. Student , Machine Learning Department, Carnegie Mellon University
https://keeganharris.github.io/

In-context (supervised) learning is the ability of an LLM to perform new prediction tasks by conditioning on examples provided in the prompt, without any updates to internal model parameters. Although supervised learning is an important capability, many applications demand the use of ML models for downstream decision making. Thus, in-context reinforcement learning (ICRL) is a natural next frontier. 

In this talk, we investigate the extent to which contemporary LLMs can solve ICRL tasks. We begin by deploying LLMs as agents in simple multi-armed bandit environments, specifying the environment description and interaction history entirely in-context. We experiment with several frontier models and find that they do not engage in robust decision making behavior without substantial task-specific mitigations. Motivated by this observation, we then use LLMs to explore and exploit in silos in various (contextual) bandit tasks. We find that while the current generation of LLMs often struggle to exploit, in-context mitigations may be used to improve performance on small-scale tasks. On the other hand, we find that LLMs do help at exploring large action spaces with inherent semantics, by suggesting suitable candidates to explore. 

This talk is based on joint work with Alex Slivkins, Akshay Krishnamurthy, Dylan Foster, and Cyril Zhang. 

— 

Keegan Harris is a final-year Machine Learning PhD candidate at CMU, where he is advised by Nina Balcan and Steven Wu, and does research on machine learning for decision making. He has been recognized as a Rising Star in Data Science and his research is supported by an NDSEG Fellowship. He is also the head editor of the ML@CMU blog. Previously, Keegan spent two summers as an intern at Microsoft Research and graduated from Penn State with BS degrees in Computer Science and Physics. 

In Person and Zoom Participation.  See announcement.

Event Website:
http://www.cs.cmu.edu/~aiseminar/

Add event to Google
Add event to iCal