Doctoral Speaking Skills Talk - Yiran Lei

— 1:00pm

Location:
In Person - Reddy Conference Room, Gates Hillman 4405

Speaker:
YIRAN LEI, Ph.D. Student, Computer Science Department, Carnegie Mellon University
https://www.yiranlei.com/

Scheduling All-to-All communications efficiently is fundamental to minimizing job completion times in distributed systems. Incast and straggler flows can slow down All-to-All transfers; and GPU clusters bring additional straggler challenges due to highly heterogeneous link capacities between technologies like NVLink and Ethernet. Existing schedulers all suffer high overheads relative to theoretically optimal transfers. Classical, simple scheduling algorithms such as SpreadOut fail to minimize transfer completion times; modern optimization-based schedulers such as TACCL achieve better completion times but with computation times that can be orders of magnitude longer than the transfer itself. 

This paper presents FLASH, which schedules near-optimal All-to-All transfers with a simple, polynomial time algorithm. We prove that, so long as intra-server networks are significantly faster than inter-server networks, FLASH approaches near-optimal transfer completion times. We implement FLASH and demonstrate that its computational overheads are negligible, yet it achieves transfer completion times that are comparable to state-of-the-art solver-based schedulers. 

Presented in Partial Fulfillment of the CSD Speaking Skills Requirement

Add event to Google
Add event to iCal