Doctoral Thesis Proposal - Alexander Koujianos Goldberg

— 3:30pm

Location:
In Person and VIrtual - ET - Traffic21 Classroom, Gates Hillman 6501 and Zoom

Speaker:
ALEXANDER KOUJIANOS GOLDBERG , Ph.D. Student, Computer Science Department, Carnegie Mellon University
https://akgoldberg.github.io/

Improving Decision-Making from Distributed Human Evaluations

Socially important decisions—from scientific funding, to college admissions and job hiring—rely on ratings or rankings supplied by multiple human evaluators. These judgments are prone to noise, bias, and strategic manipulation, and there is seldom an objective ground truth against which to determine their quality. The goal of this thesis is to understand and mitigate such errors in distributed human evaluation in order to make better decisions. Towards this end, we both conduct controlled experiments in review processes and develop principled algorithms with provable guarantees. 

In particular, we conduct large-scale experiments at peer review conferences to expose sources of error in evaluation and identify opportunities for improvement. Then, we develop a method for selecting top candidates on the basis of uncertain evaluations, providing a principled instantiation of a "peer review lottery." Finally, we design privacy-preserving algorithms for releasing anonymized time-series and graph data, which can enable more transparency into review processes while preserving participant anonymity.

Thesis Committee

Giulia Fanti (Co-chair)
Nihar B. Shah (Co-chair)
Tom Mitchell
John Ioannidis (Stanford University)

Additional Information

In Person and Zoom Participation.  See announcement.

Add event to Google
Add event to iCal