Accessibility Lunch Seminar - Prerna Khanna

— 2:00pm

Location:
In Person and Virtual - ET - Newell-Simon 3001 and Zoom

Speaker:
PRERNA KHANNA , Ph.D. CandidateDepartment of Computer ScienceStony Brook University
https://sites.google.com/view/prerna-khanna

Making Smartphones Accessible to Blind Users Using Wearables

Blind users interact with their smartphones predominantly using screen readers such as TalkBack for Android and VoiceOver for iOS to read out content and navigate touchscreen interfaces. However, touchscreen interactions are challenging for blind users due to the need for two-handed interactions (conflicting with cane or guide dog use), vulnerability to shoulder-surfing attacks, and gesture overloading. This talk explores how we can overcome these challenges by moving core interactions beyond the screen and leveraging gestural input from commodity wearables.


I will first present AccessWear, a system-level framework that uses novel input virtualization to seamlessly map smartwatch gestures to touch events. This approach enables blind users to control any mobile application without requiring a single line of code change from developers. Our central premise is that while wearable gestures enable touch-screen-free input, existing recognition models fail for blind users. These models are designed and trained on sighted users, and fail to capture the unique motion properties of blind users. This gap calls for specialized algorithms that reflect how blind users perform gestures in everyday settings. Next, I will show how we characterized the unique motion properties of blind user gestures and used these insights to develop a robust, specialized recognition approach that works without per-user training.

Building on this foundation, I extend gestural input beyond simple navigation to support complex, fine-grained interaction tasks for blind users. With GestureVoice, I will show how gestures can be combined with voice commands to support fine-grained interactions such as text editing, which is one of the most difficult tasks for blind users on mobile devices. This multimodal system reduces text editing time significantly for blind users. 

For More Information:
pmowar@andrew.cmu.edu

Add event to Google
Add event to iCal