Roger Iyengar

Scaling Up Wearable Cognitive Assistance for Assembly Tasks Degree Type: Ph.D. in Computer Science
Advisor(s): Mahadev Satyanarayanan (Satya)
Graduated: May 2023

Abstract:

Wearable Cognitive Assistance (WCA) applications run on wearable mobile de- vices, to provide guidance for real world tasks. Physical assembly tasks have been a significant focus of research on WCA. We introduce new techniques to support the development of WCA applications for more complex assembly tasks than previous techniques supported. In addition, our work reduces the load on developers creating WCA applications by eliminating the need to collect and label real training images. We accomplish this by training computer vision models on synthetically generated images. This dissertation investigates escalation to human experts in cases when a user is not satisfied with the automated guidance from an application. Lastly, we develop a new version of a software framework for WCA applications, and evaluate ways in which WCA applications can benefit from running computations directly on mobile devices.

Thesis Committee:
Mahadev Satyanarayanan (Satya) (Chair)
Martial Hebert
Roberta Klatzky
Padmanabhan Pillai (Intel Labs)

Srinivasan Seshan, Head, Computer Science Department
Martial Hebert, Dean, School of Computer Science

Keywords:
Wearable Cognitive Assistance, Wearable Computing, Edge Computing, Split Computing, Computer Vision, Synthetic Data