VASC Seminar - Mike Shou

— 4:30pm

Location:
In Person - Newell-Simon Hall 3305

Speaker:
MIKE SHOU , Assistant Professor / Presidential Young Professorship, Department of Electrical & Computer Engineering, National University of Singapore
https://sites.google.com/view/showlab

Video intelligence in the era of multimodal

The past few years have witnessed great success in video intelligence, as supercharged by multimodal models. In this talk, I will start with a brief sharing of our efforts, in building video-language models for understanding and diffusion models for video generation. Yet, video understanding and generation have always been two separate research pillars, despite their strong synergy. This motivates us to develop Show-o, one unified single transformer that can do both multimodal understanding and generation. Show-o is the first to unify autoregressive and discrete diffusion modeling, flexibly supporting a wide range of vision-language tasks of any input/output format, including visual question-answering, text-to-image/video generation, and generation of video keyframes with captions, all within one single 1.3B transformer. Show-o sheds light for building the next-generation multimodal video models. 

— 

Mike Shou is an Assistant Professor under Presidential Young Professorship at National University of Singapore. He was a Research Scientist at Facebook AI in the Bay Area. He obtained his Ph.D. degree at Columbia University with Prof Shih-Fu Chang. His research mainly focuses on video and multimodal. He received the Best Paper Finalist at CVPR 2022, Best Student Paper Nomination at CVPR 2017, EgoVis Distinguished Paper Award 2022/23. His team won 1st place in the international challenges including ActivityNet, EPIC-Kitchens, Ego4D. He is a ST Engineering Distinguished Professor and a Fellow of National Research Foundation Singapore. He is on the Forbes 30 Under 30 Asia list.

Event Website:
https://www.ri.cmu.edu/event/video-intelligence-in-the-era-of-multimodal/


Add event to Google
Add event to iCal