HoloToolkit is a powerful open-source framework brought to you by Microsoft that encapsulates key concepts of working with HoloLens: Gaze, Gesture, Voice, Spatial Audio and Spatial Mapping. HoloToolkit can be easily extended in meaningful ways to help the growing community of HoloLens developers avoid boilerplate code and focus on their apps.
MICROSOFT HOLOLENS PROJECT X-RAY MIXED REALITY EXPERIENCE BRINGS ALIEN ROBOTS AND PLASMA BEAMS INTO YOUR LIVING ROOM
Today, I had the privilege of experiencing a demo of Microsoft HoloLens Project X-Ray at a Seattle developer event, and I am excited to share that it establishes a high bar for Mixed Reality experience design, while bringing alien robots right into your living room!
I'd like to congratulate the HoloLens Project X-Ray team on such an awesome achievement, and I can't help but share my experience with the world.
Upon arriving to the Microsoft HoloLens event, I was signed in by friendly staff and directed into a waiting area where I was shown an introductory video on how to use a HoloLens headset, along with a few other people from my demo group. A young lady quickly took my IPD measurements with a device that looked like something out of an optometrist's office, after which I spent some time waiting for my time slot, chatting with other folks about how they got to the event, and sharing our expectations for the upcoming demo.
After hanging out a bit in the waiting area, I was directed to a setup which looked like a regular living room that anyone could have in their own home. Even though Microsoft was showing demos to several people simultaneously, the demos were individual, with only one person in a room at a time. Inside the room, another person entered my IPD measurements into the system, gave me a HoloLens headset to put on, briefly explained how to adjust it, what to expect from the demo, and how to do the air tap gesture to interact with the augmented world by putting together my thumb and index finger to tap the air in front of me.
The HoloLens device itself felt fairly light on my head, and while I couldn't tell whether the lens was made of glass or a strong polycarbonate material, it felt and looked really solid. The headset looked and felt like a polished consumer product rather than a technical prototype, kudos to the Microsoft HW team on great industrial design!
After making very small adjustments, such as tightening the headband and moving the lenses a bit closer to my face, I was launched into the Project X-Ray experience.
The first thing I saw was an interaction reticle in the middle of my field of view, which looked like something from Tony Stark's Iron Man suit interface, and which followed my gaze wherever I looked. It consisted of several concentric circles separated from each along the z axis, and for a moment, the circles formed a tunnel that directed my attention to on-screen dialogs hovering in 3D space. The dialog borders and text were really crisp, and I got a sense for the high resolution of the holographic display.
The interaction reticle felt like it had a life of its own, changing and breathing wherever I looked, and I’d like to share more on that later.
After air tapping to start the experience, the system prompted me to turn left and right, and I saw a red 3D triangulated surface outline showing on top of the walls as I was scanning them. After air tapping again, the wall in front of me literally opened up, with some light coming out, and a small robot creature emanated from the opening. The robot turned out to be a guide who told me about enemy robots, and that I needed to point the reticle at the robots and air tap to shoot them down.
The robot guide disappeared, and I saw the wall in front of me starting to show some cracks, after which metal tentacles suddenly burst through. The tentacles unfolded outwards, showing a dark opening in the middle, through which enemy robots started to fly out, hovering a few feet above the floor. I did several air tap gestures in front of me, which resulted in plasma beams shooting out as if I was wielding a weapon in a modern futuristic first person shooter. I dodged fireballs that the robots threw at me as I was shooting them down with the plasma beam, and it felt natural and satisfying.
Red arrows surrounding the center of the interaction reticle showed the direction of enemy attacks, and I naturally turned to look around in the direction of the arrows. I saw more cracks in the walls, and more metal tentacles burst through. I defended myself against several robot waves, after which gameplay intensified, with robots moving and shooting more aggressively, and hovering robots being replaced by crawling robot scorpions. I nearly forgot being instructed to say "x-ray", but as I did so, I enabled x-ray vision and started seeing robots right through walls, so I could shoot them down before they crawled out.
The final battle was a large boss robot that aggressively moved back and forth, shooting fireballs and spawning its robot minions. I shot down a few minion waves, not forgetting to shoot at the boss in between. The big robot exploded in the air, and I saw my final score: 11550, while I was told that the average was around 8000, so I did fairy well :)
The whole experience lasted for about 10 minutes, and even though I just experienced an alien robot invasion, it always felt like I was still in the room, a true Mixed Reality experience.
I took off the headset, found myself transported back to the less exciting reality with no holograms, but I have to say that the field of view that some people talked about after the Build conference was never a problem throughout the experience.
Prior to putting on the HoloLens for the first time, I read about its field of view, but it was never an issue on the device I wore at the event. The characters and objects felt like they were a part of the surrounding physical space, and the whole experience was seamless. Virtual objects were augmenting the real world, with metal tentacles bursting right through walls, and with alien robots crawling on and beneath the walls that I could see through with my x-ray vision super-powers. I didn't notice screen borders and didn’t have to think about the field of view.
In summary, I put the HoloLens on and started shooting alien robots right in that room, while the HW got out of the way, which felt really natural and seamless.
After the demo, some people from the HoloLens development team showed us a brief presentation on the features of the HoloLens developer SDK, provided answers to a few developer questions, and a timeframe for devkit availability, which was already announced at a recent event in New York: Q1 2016, $3000, I'll take two!
It's great to see Microsoft raising the bar and establishing a new gold standard for Augmented and Mixed Reality interaction design, but more on that next time.