Computer Science 5th Year Masters Proposal

Friday, May 8, 2020 - 1:30pm to 2:30pm


Virtual Presentation Remote Access Enabled - Zoom


JUNHAN (JUDY) KONG, Masters Student /JUNHAN%20%28JUDY%29%20KONG

An Authoring Tool for Creating Interactive AR User Tutorials by Demonstration

Using complex interfaces has been shown to be challenging, especially for certain user groups. Existing tutorial systems can be cumbersome, and sometimes difficult to use. While augmented reality has been adopted to help users learn to use some interfaces, these AR user manuals are usually designed specifically for one single interface, so creating AR user manuals for a variety of interfaces requires a substantial amount of work, and the authoring tools for these manuals are usually not friendly to a general user. To solve this problem, we present a system that automatically generates interactive AR user tutorials by demonstration. Our system first guides experienced users to add references that help our system understand the interface and to demonstrate steps to complete certain tasks. Then our system automatically detects user interactions and asks users for supplemental information on the interactions. Based on this information, our system generates a sequence of user actions to perform for each task, and provides a combination of text instructions, visual and audio guidances for new users to access the interface.

Thesis Commitee:
Jeffrey P. Bigham (Chair)
Patrick Carrington

Additional Proposal Information

Zoom Participation Enabled.  See announcement.

For More Information, Contact:


5th Year Masters Proposal