Jigsaw (2022)

How can we go beyond pixels and blur the boundary between what’s physical and what’s virtual?

This was my internship project at Snapchat. In Jigsaw, we explored immersive storytelling with AR and IoT devices. The system enables anyone to make and experience 4D-theatre-like stories at home using common IoT devices like smart fans, lights, and speakers. Now, story effects can be both visual and tangible.

We demonstrate how ubiquitous computing and augmented reality can merge to yield a world where computing happens not just inside screens but everywhere in your room.

Check out our presentation for more details: Google Slides.

Publication

Lei Zhang, Daekun Kim, Youjean Cho, Ava Robinson, Yu Jiang Tham, Rajan Vaish and Andrés Monroy-Hernández. 2024. Jigsaw: Authoring Immersive Storytelling Experiences with Augmented Reality and Internet of Things. Under submission to CHI 2024.

What I’ve done

Story ideation; Interaction design for peer-to-peer participant character registration; Developed the story experiences connecting AR and IoT devices; Wrote sections of the paper.