Doctoral Speaking Skills Talk - Kevin Kuo
April 29, 2026 11:30AM—12:30PM
Location:
1301
-
Collaborative Innovation Center
Speaker:
KEVIN KUO,
Ph.D. Student, Computer Science Department, Carnegie Mellon University
https://imkevinkuo.github.io/
To improve trust and adoption of collaborative learning systems, it is important to provide opt-out guarantees which enable removal of unwanted (e.g. harmful or private) information after it has been used to train a model. A popular solution to this problem is approximate unlearning, which efficiently updates an LLM so that it behaves (roughly) as if it was not trained on a subset of data to begin with. However, existing methods are brittle in practice and can easily be attacked to reveal supposedly unlearned information. To alleviate issues with approximate unlearning, we instead propose SIFT-Masks (SIgn-Fixed Tuning-Masks), an exact unlearning method based on model merging.
SIFT-Masks addresses two key limitations of standard model merging: (1) merging a large number of tasks can severely harm utility; and (2) methods that boost utility by sharing extra information across tasks make exact unlearning prohibitively expensive. SIFT-Masks solves these issues by (1) applying local masks to recover task-specific performance; and (2) constraining finetuning to align with a global sign vector as a lightweight approach to determine masks independently before merging. Across four settings where we merge up to 500 models, SIFT-Masks improves accuracy by 5-80% over naive merging and uses up to 250x less compute for exact unlearning compared to other merging baselines.
Presented as part of the CyLab Student Seminar
Contact
Matt Stewart