Computer Science Thesis Proposal
— 1:30pm
Location:
In Person
-
Traffic21 Classroom, Gates Hillman 6501
Speaker:
DRAVYANSH SHARMA
,
Ph.D. Student, Computer Science Department, Carnegie Mellon University
http://www.cs.cmu.edu/~dravyans/
Data-driven algorithm design for principled hyperparameter tuning in machine learning
For any new machine learning technique, a large body of research often follows up in order to tune the technique to work suitably for each of the numerous application areas, requiring significant scientific and engineering efforts. Moreover, this typically involves unprincipled approaches for hyperparameter selection without any guarantee on global optimality. This thesis is inspired from the recently proposed paradigm of ‘data-driven algorithm design’. We show how to tune some core machine learning algorithms with formal near-optimality guarantees in statistical and online learning settings.
Given multiple problem instances of a learning problem from some problem domain, we develop approaches to learn provably well-tuned parameters over the domain and answer questions related to the number of problem samples needed to learn a well-tuned learning algorithm. More precisely, our approaches apply to the following diverse scenarios:
-
selecting graph hyperparameters in semi-supervised learning,
-
setting regularization coefficients in linear regression,
-
controlling the robustness vs. abstention trade-off using parameterized nearest-neighbor algorithms,
-
meta-learning common parameters for similar tasks, and
-
learning adaptively in changing environments.
In addition to providing techniques for tuning fundamental learning algorithms, we also develop tools applicable to data-driven design more generally.
Thesis Committee:
Maria-Florina Balcan (Chair)
Tom M. Mitchell
R. Ravi
Avrim Blum (Toyota Technological Institute at Chicago) (TTIC)
Tim Roughgarden (Columbia University)
Additional Information