Minecraft IRL
By far, the most impressive demo for my money was the Minecraft demo — though Microsoft called it something like "Building Blocks" or some such, presumably so as not to fully commit to releasing a full holograph version of Minecraft. But before we could enter this virtual world — actually, the virtual entered our world — we had to strap on the development unit for the HoloLens.
It's a contraption, to be sure. There's a small, heavy block you hang around your neck which contains all the computing power. It's comprised of lenses and tiny projectors and motion sensors and speakers (or something that makes sound, anyway), and god knows what else. And then there's a screen right there in your field of view.
"It's just sitting there, perfectly flat on the table, reacting in space to your head movements"
A "screen in your field of view" is the right way to think about HoloLens, too. It's immersive, but not nearly as immersive as proper virtual reality is. You still see the real world in between the virtual objects; you can see where the magic holograph world ends and your peripheral vision begins.
But before you can apply your jaded "I've done VR before" attitude to this situation, you look down at the coffee table and there's a castle sitting right on the damn thing. It's not shimmery, but it's not quite real, either. It's just sitting there, perfectly flat on the table, reacting in space to your head movements. It's nearly as lifelike as the actual table, and there's no lag at all. The castle is there. It's simply magic.
You definitely have a big stupid grin on your face even though the contraption that's strapped to it is pressing your eyeglasses into the bridge of your nose in a painful way.
Then it's demo time. You can't touch anything, but you can look and point a little circle at objects on it by moving your head around. You learn how a "glance" is just you looking at things and pointing your reticle at them, and an "AirTap" is the equivalent of clicking your mouse. The demo involves digging Minecraft holes and blowing up Minecraft zombies with Minecraft TNT. It's basically incredible to see these digital things in real space.
You blow up a hole in the table and then you look through it to more digital objects on the floor. You blow up a hole in the wall, and tiny bats fly out. You see that behind your very normal wall is a virtual hellscape of lava and rock. You peer into the hole, around the corner, and see that dark realm extend far into space.
Skype
Microsoft’s Skype demo was as equally impressive to me as playing around with Minecraft blocks in a living room. After a two-hour keynote, Microsoft wanted me to fix a light switch. It all started by sitting down and facing some tools and a socket with exposed wiring. A little dazed and confused, I looked up and scanned across the Skype interface which was suddenly appearing in front of me, and picked a face to call. The video call popped into a little window, and my journey to fix a light switch began.
On the other end of the call was a Microsoft engineer. I could see and hear her, but she could only hear me and see exactly what I was seeing in front of me. My eyes, or the headset on my head, was relaying everything over Skype. It was a support call of sorts — here she was to help me fix a light switch. We started by pinning her little window on top of a lamp. I could then look around the room and return to the lamp to see her face. She guided me where to go. It felt strangely natural, and I didn’t need to configure anything or learn gestures other than the same "Air Tap" you use to simulate a mouse click.
While I was being talked through which real world tools we needed for the job, the Microsoft engineer called my attention to the wall with wiring and then started drawing where to position the light switch right on the wall. Thinking about it now it sounds totally surreal, but during the demo I didn’t even think about it — it just felt like I was being guided around with annotations and a helpful friend. We connected the wiring, tested it for an electrical current, and then turned the power back on and switched the light on. It was all fixed, and all by using a crazy combination of a headset, augmented reality, and Skype. It might sound gimmicky, but the applications here are truly impressive. I use YouTube guides to figure out home improvements or to service my car, but this is on another level. Imagine a surgeon performing complex surgery and writing notes in real time and guiding a colleague through it all. Imagine support calls to resolve a problem with your PC. If this works as well as Microsoft’s controlled demo, then this really has the ability to change how we communicate and learn.
Holo Studio
Microsoft's next demo didn't have us using the HoloLens prototypes directly. Instead, we watched as "Nick" (nobody in Microsoft's blue-tinted demonstration basement has last names. I asked.) manipulate objects in digital space so he could build a Koala bear or a pickup truck. It was actually quite impressive, as cameras filmed him and screens showed both Alex and the virtual objects he was manipulating in the same space in real time.
The idea was to convince us that HoloLens would unleash a wave of creators who would be able to dream up 3D objects with little to no training. It's much easier to understand what a thing is in your living room than it is in AutoCad.
Microsoft HoloLens
But sitting there after our whirlwind of actually experiencing HoloLens, my mind was elsewhere. For example, there are only a few ways to interact with this system so far:
Glance: you point your head at something.
AirTap: you make a "Number 1" sign with your hand, then move your finger down like you're depressing a lever.
Voice: you can issue commands, usually to switch what "tool" you're using.
Mouse: So actually the neatest thing is that objects you use to interact with computers can be used to interact with holograms.
That seems like enough, but it's not nearly enough. It's wildly impressive that these objects really do feel like they're out there in your living room, but it's equally depressing to know that you can't treat them like real objects.
At one point in the demo, Alex needed to put a tire on his pickup. He had to twist his body and head around to get his pointer in just the right spot and get the tire arranged just right to fix on the axle. Then, AirTap! the tire is connected. But how much easier would it be if you could grab the tire in your actual hands?
Our hands are simply more dextrous than our necks. You have finer control over small motions, you can move your hands in so many different ways and vectors, with pressure, nuance, and delicacy. Your neck and head, well, not so much.
But then Microsoft gave us 3D printed Koalas with a USB drive inside them, which was nice. And if this HoloLens thing takes off, you will be able to design your own and it will be way easier than learning current 3D design software. But not as easy as it would be if you just imagined building with holograms.
Walking on Mars
Microsoft has teamed up with NASA to let scientists explore what Curiosity sees on Mars. Instead of panoramic imagery on a computer screen, Microsoft’s demo lit up a room and turned it into Mars. I walked around the rocky terrain, bumped into the Curiosity rover, and generally just checked out a planet I will never visit in my lifetime. It’s a totally new perspective that felt like I was immersed in touring Mars, but not necessarily there. The field of view felt a little too limited to truly immerse myself and trick my brain into thinking I was really on another planet, but what impressed me most is what Microsoft has built into this experience.
I held a call with a NASA engineer, and he talked me through the terrain. I squatted to look more closely at rocks, took snapshots of various rock formations, and even planted flags for points of interest. My jaw dropped when I ventured over to a PC in the room and started to experiment with the mouse. I pulled the mouse pointer off the screen and suddenly it was on the floor next to me, allowing me to set markers in the virtual environment. It’s everything I’ve seen in demonstrations from Microsoft Research before, but here it was on my head and working.
The collaboration part was the key here, allowing me to interact with this data in a unique way, but also alongside the NASA engineer who could drop flags on the Mars terrain and guide me to look at certain sections. While this isn’t traditional productivity with a mouse and keyboard, it’s certainly something new and intriguing. I could see this type of scenario working for big teams that need to communicate across time zones and on big sets of complex data.
Overall, HoloLens is Microsoft at its most ambitious. It’s a big bet on the future of computing, the future of Windows, and ultimately the future of Microsoft itself. While the company is struggling at mobile, it wants to catch the next wave of computing and lead. Is HoloLens the next wave? Developers and consumers will be the ultimate test of that, but if anything HoloLens is an incredibly brave and impressive project from Microsoft. It’s true innovation, which is something Microsoft has lacked during its obsession with protecting Windows. It’s also another example of an experience that takes the complex technology out of the way, leaving you to experience what really matters.
