RoomAlive is a proof-of-concept prototype that envisions a future of interactive gaming with projection mapping. RoomAlive transforms any room into an immersive, augmented entertainment experience through the use of video projectors. Users can touch, shoot, stomp, dodge and steer projected content that seamlessly co-exists with their existing physical environment. RoomAlive builds heavily on our last research project, IllumiRoom, which explored interactive projection mapping surrounding a television screen. IllumiRoom was largely focused on display, extending traditional gaming experiences out of the TV. RoomAlive instead focuses on interaction, and the new kinds of games that we can create with interactive projection mapping. RoomAlive looks farther into the future of projection mapping, and asks what new experiences will we have in the next few years?

The core building block of RoomAlive is a projector depth-camera unit, also known as a procam. These consist of a commodity wide field of view projector, a Microsoft Kinect sensor and a computer. The projector is used for display and the Kinect is used for tracking. Normally video projectors are used to display PowerPoint presentations on flat screens. RoomAlive uses cheap commodity video projectors to animate every square inch of your living room, this technique is known as projection mapping.

With one procam, users can have an IllumiRoom style experiences. If you have six procam units, you convert every square inch of your living room into an interactive display.

Every square inch of your living room is a pixel, both for display and for input

For most projection mapping pieces, content must be painstakingly created for the particular object to be projected on. The artist must take into account the object shape and color, along with the projector location, field of view etc. RoomAlive enables game designers to create projection mapped games independent of the particular room that the content is displayed in. All the content in RoomAlive is driven in real-time and dynamically adapts to the exact color and geometry of the user’s living room. For instance, RoomAlive can automatically detect the floors and walls of the room

The procam units are as easy to install as a light fixture. Most projection mapping software requires the designer to calibrate the system by clicking on an image or waving a checkerboard. With RoomAlive, each procam unit is individually auto-calibrating and self-localizing. This means that the user just installs the fixture, and then a unified model of the room across all six procams is created with no user intervention.

In order to display images that look 3D with the correct perspective, RoomAlive tracks the user’s head position in the room (across the six Kinects) and renders a view-dependent projection essentially making. Similar to the parallax effect in the Amazon Fire phone, RoomAlive content looks correct as the user moves their head around.

Credits

This was a large group effort conducted at Microsoft Research by Brett Jones, Raj Sodhi, Mike Murdock, Ravish Mehra, Hrvoje Benko, Andy Wilson, Eyal Ofek, Blair MacIntyre, & Lior Shapira.

Paper: Download the paper

Reference: Jones, B., Sodhi, R., Murdock, M., Mehra, R., Benko, H., Wilson, A. D., Ofek, E., MacIntyre, B., Shapira, L. RoomAlive: Magical Experiences Enabled by Scalable, Adaptive Projector-Camera Units. ACM UIST, 2014.

Download all high res images: here.