How HoloLens Affects the Future of Projection Mapping

How HoloLens Affects the Future of Projection Mapping
January 28, 2015 Rajinder Sodhi

Editor’s Note: I’ve worked at Microsoft Research and personally know some of the awesome designers and engineers on HoloLens.

The question I’m always asking myself of any new invention is “can I see myself using this?” And if not, “how do I see it used, where are its killer applications?” In truth, whenever I see new technology, I can’t help but also wonder how it affects the projection mapping world. It’s the light enthusiast in me. We’ve seen some amazing inventions in the last few years (e.g., Oculus, Kinect, Sphero), and while the mainstream perception is usually a supportive “yes,” the answer is typically a “no.”

It’s always amazing how often companies gear their marketing pitch towards gaming. This is one of the biggest challenges of a new invention. You have to make a well-informed guess at which market you think a new technology will best serve and it has to be big, especially when it comes from a large company. As XBox boss Phil Spencer says, “there’s not a successful consumer electronics device on the planet where gaming is not a primary app category.” Take for instance, the Kinect, which made it’s way into the hands of thousands of gamers in 2011, following the success of the popular Wii console. The goal was to revolutionize gaming, and for a while, their marketing pitch had most of us saying “Yes!” While there are some truly impressive Kinect games, the reality is that it’s been struggling catching on as a gaming interface.

HoloLens

In reality, new technologies are nice to look at, but change via new platforms, new interfaces, etc. can take a long time to hit widespread adoption. There are a lot key factors at play like timing, accessibility and chief among them are keeping the user experience and design at the center of attention. There are a lot of new inventions out there looking for a user, that are less user-driven, but still have a lot of potential.

With all that said, I’m taking a cautiously optimistic view of HoloLens. If you missed the pitch, I am referring to the new invention Microsoft introduced at the Windows 10 event last week. Lead designer Alex Kipman and team at Microsoft have been working on the HoloLens for nearly 6 years. Note, Alex was also the lead designer and developer of the Kinect. Hit the jump (above) to see the HoloLens’ various applications and check out the live demo below.

In short, the Augmented Reality (AR) device projects light onto an eyepiece worn by a user. The user can still see parts of the real world, but the display simulates seeing a real hologram in space. Note that everyone has to wear one of these devices to see the hologram. The HoloLens isn’t just a concept either, it actually appears that Microsoft has a full fledged demo (they even have YouTube advertising) to back up their claims. The brains of the headset include a CPU, GPU and a “holographic processing unit.” The system can track eye movements, process voice commands and can even follow hand gestures like the Kinect. The list of amazing engineering feats go on and on, but the video also shows off high level concept experiences as working (albeit works in progress) demos verified by several prominent media outlets. So this isn’t hogwash and the live demo stream looked most impressive.

The HoloLens isn’t the first device to enter the realm of AR. The AR hologram vision has been floating around in the research community for decades upon decades, dating all the way back to Ivan Sutherland (1965), “The Ultimate Display.”

sutherland

More recently, the wearables space has been commercially occupied by the likes of Google Glass, Vuzix (see video below) and Oculus, with new players like Magic Leap and Meta’s Space Glasses hot on their tails. The HoloLens incarnation of the vision is the most practical variant we’ve seen up to this point.

So how does this actually affect projection mapping? Well friends, if you take the technology to its limit and assume you can display anything on, and above your real environment through your glasses, and you also assume that everyone is wearing a headset, you are essentially doing what projection mapping can do today and more. There are some pretty fun ideas that have a lot of overlap with projection mapping (room sized games, interior design, brilliant animations, etc.).

Maui

MineCraft

motorcycle

So what’s the downside? Amidst all of the internet buzz, devices like HoloLens have a lot of fundamental practical challenges. For instance, it’s hard not to think about the bulk, weight, resolution and latency among others issues with wearable glasses. Not to mention, everyone has to wear one in order for users to have a shared experience. I have a similar opinion with the Oculus Rift via the latest incarnation of the VR movement. The number of people (including hardcore gamers) who tell me they’d never wear something as big as the Oculus on their head is still pretty astounding. This is one of the key benefits of projection mapping: large immersive experiences that bring together thousands of people without encumbering the user with a large device. Of course, every technology has its downsides and projection mapping is no different. You still fundamentally need a surface to project on which makes it difficult to achieve the holographic look that HoloLens so nicely espouses.

Devices like the HoloLens are a true marvel of engineering with an immense amount of potential, and quite honestly, I think Microsoft is really on to something. We have good sources telling us that technical challenges like tracking are being solved by the best researchers in the world with amazing results. In answering my very first question, I always try to think about how a new invention practically works now, and then take it to its limit for how it will change a user experience in the future. If Microsoft can bring the technology to market while keeping it accessible to new users, we’ll be off to the races! My guess is that in the in the near term, the device’s disadvantages will cause it to pivot out of the end consumer market and into more business niche applications, following the same pattern as Google Glass. So for now, we have to play the waiting game. Stay steady projection mappers!