Computer Science Teaching Track Candidate
Location:
In Person and Virtual - ET
-
Reddy Conference Room, Gates Hillman 4405
Speaker:
OMAR KHATTAB
,
Ph.D. Candidate, Computer Science Department, Stanford University
https://omarkhattab.com/
Building More Reliable and Scalable AI Systems with Language Model Programming
It is now easy to build impressive demos with language models (LMs) but turning these into reliable systems currently requires complex and hand-tuned combinations of prompting, chaining, and fine tuning LMs.
In this talk, I present LM programming, a systematic way to address this by improving four layers of the LM stack. I start with how to adapt LMs to search for information most effectively (ColBERT, ColBERTv2, and UDAPDR) and how to scale that to billions of tokens (PLAID), which is work that has sparked applications at dozens of companies and research labs. I then discuss the right architectures and supervision strategies (ColBERT-QA, Baleen, and Hindsight) for allowing LMs to search for and cite verifiable sources in their responses. This leads to DSPy, a programming model that replaces ad-hoc techniques (for prompting, chaining, and finetuning LMs) with composable modules and automatic optimizers (which act on high-level function signatures). Even simple DSPy programs routinely outperform standard pipelines with hand-created prompts, potentially using relatively small LMs.
I conclude by discussing how the framework established by DSPy enables a new degree of research modularity, one that stands to allow open research to again lead the development of AI systems.
—
Omar Khattab is a fifth-year CS Ph.D. candidate at Stanford NLP and an Apple Scholar in AI/ML. He is interested in Natural Language Processing (NLP) at scale, where systems capable of retrieval and reasoning can leverage massive text corpora to craft knowledgeable responses efficiently and transparently. Omar is the author of the ColBERT retrieval model, which has helped shape the modern landscape of neural information retrieval (IR), and author of several early multi-stage retrieval-based LM systems like ColBERT-QA and Baleen. Omar's work has been funded by an Apple Fellowship and Eltoukhy Family Graduate Fellowship, and his research has received industry grants from Azure, IBM, Oracle, and Virtusa. His recent work includes the DSPy programming model for building and optimizing reliable language model systems—by bringing structure, composition, and new forms of optimization into the space of prompting, finetuning, and chaining retrieval and language models. Much of Omar's work forms the basis of influential open-source projects, and his lines of work on ColBERT and DSPy have sparked applications at dozens of academic research labs and leading tech companies, including at Google, Meta, Amazon, IBM, VMware, Baidu, Huawei, AliExpress, and many others. Omar obtained his B.S. in Computer Science from CMU-Qatar.
Faculty Host: Iliano Cervesato I
n Person and Zoom Participation. See announcement.