Everywhere Cursor (2021-2023)
What if your entire room became a part of your desktop or laptop environment?
Everywhere Cursor explores cross-device interaction between desktops and AR. With it, we intend to connect the “boring” (but still arguably the most useful) realm of technology with the world around it. This mouse technique extends the behaviour of mouse cursors from desktop monitors to any arbitrary 3D surfaces, based on an algorithm inspired by how ants crawl around spaces. One of the main challenges was in handling behaviours from navigating a non-Euclidean, discontinuous surface. Check out the paper to see how we solved this issue.
What if drag-and-drop really meant dropping of the document to the printer in the room?
The entire room (e.g., desk) is now an extension of the computer.
Designing projector mapping decorations using the mouse on a complex, jagged surface.
Unity rendering of the cursor and the environment. Our algorithm uses legs and circular raycast to imitate an ant crawling its surrounding surfaces.
Credits
Dr. Dan Vogel and Nikhita Joshi for their guidance in iterating on the mouse technique design and paper writing.
Publications
Daekun Kim, Nikhita Joshi, and Daniel Vogel. 2023. Perspective and Geometry Approaches to Mouse Cursor Control in Spatial Augmented Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23), April 23–28, 2023, Hamburg, Germany. ACM, New York, NY, USA, 19 pages. https://doi.org/10.1145/3544548.3580849
Daekun Kim and Daniel Vogel. 2022. Everywhere Cursor: Extending Desktop Mouse Interaction into Spatial Augmented Reality: Extended Abstract. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA '22). Association for Computing Machinery, New York, NY, USA, Article 389, 1–7. https://doi.org/10.1145/3491101.3519796
Press
CTV News, 'Blurring the line between the virtual and the physical word': Waterloo researchers helping develop the metaverse, Apr. 2022
CBC News, These University of Waterloo professors are helping build the metaverse, Apr. 2022
What I’ve done
Made hardware and software adjustments to the projection mapping system; Devised the cursor mapping algorithm; Developed the experiment and demo software in Unity; Ran a 20-person user study and performed data analysis; Wrote most of both papers.