Doctoral Thesis Oral Defense - Alexander Koujianos Goldberg
May 29, 2026 1:00PM—3:00PM
Location:
4405 & Zoom
-
Gates and Hillman Centers
Speaker:
ALEXANDER KOUJIANOS GOLDBERG,
Ph.D. Candidate, Computer Science Department, Carnegie Mellon University
https://akgoldberg.github.io/
Many consequential decisions require decision makers to combine noisy judgments from distributed human evaluators, often without access to an objective ground truth. Scientific peer review and grant funding are central examples, but similar challenges arise in hiring, admissions, medical decision-making, online platforms, and AI evaluation. This thesis studies how to understand and mitigate errors in distributed human evaluation in order to make better decisions. It combines controlled experiments in real review processes with principled algorithms that provide formal guarantees.
The first part of the thesis uses two large-scale experiments at ML/AI conferences to study interventions for improving scientific peer review. One experiment examines whether meta-evaluation can improve review quality; the other studies a live deployment of a large-language-model assistant for paper authors. Together, they show both the promise and the limits of interventions aimed at improving review processes.
The second part develops algorithms for selection under uncertainty. Motivated by the growing use of lotteries in scientific funding, we formalize the goals behind randomized selection, show that existing designs often fail to meet them, and introduce efficient algorithms for randomized selection with provable guarantees.
The third and final part studies privacy-preserving data release for evaluation systems. We show how released review, time-series, and graph data can compromise participant privacy, and develop mechanisms for sharing useful data while protecting anonymity.
Thesis Committee:
Giulia Fanti (Co-Chair)
Nihar B. Shah (Co-Chair)
Tom Mitchell
John Ioannidis (Stanford University)
In-person and Zoom
Contact
Matt Stewart