Microsoft HoloLens

[Irbis]

Microsoft’s VR Experiment Is Called Windows Holographic

Microsoft takes the next step to putting digital content in your life.

Today the Windows 10 Live Event, Microsoft revealed the Windows Holographic, a new project that uses holograms to immerse users in digital content.

Microsoft's holographic technology works with a new device called Microsoft HoloLens, a fully untethered device that uses see-through lenses, spatial sound, and advanced sensors. It features a third processing unit (called "Holographic Processing Unit") in addition to a CPU & GPU.

Windows Holographic also uses a new program called Holo Studio, which lets you create your own holograms and then 3D print them. A quadcopter was created during the presentation to show off how the technology works. Yes, it can actually fly!

Windows Holographic will be available within the Windows 10 timeline, according to Microsoft.

[salm]

Great news.

[InsaneTD]

Oh that sounds fun.

[SpottedKitty]

Intriguing, but 3D systems have been tried before without much market penetration; maybe they'll get it right this time. I'm more interested, though, in seeing how the "W10 will be free for the first year" thing works out.

[Fingolfin_Noldor]

Intriguing, but 3D systems have been tried before without much market penetration; maybe they'll get it right this time. I'm more interested, though, in seeing how the "W10 will be free for the first year" thing works out.

They are probably aiming for professionals.

[Irbis]

Intriguing, but 3D systems have been tried before without much market penetration

Um, how?

This is not 3D VR system. These are trivial to make - add two screens to glasses, connect to PC, done. This is fully fledged PC (no cables) with integrated Kinect. It knows where all the walls are, what gesture you're making, where you look.

Think of implications for a second - TV? Who needs it, this can make your whole wall into a TV. Phone? Has it too, done better, even. New wall paint? Why bother, you can make your room into Palpatine office if you want. Or XIII century castle. Intelligent fridge? Save your money, this does it too. GPS? Probably built in. Laptop/tablet? This is more portable and offers better privacy. Fancy fishes in aquarium? Can be digital, or hell, even fictional. With this, every single other electronic device in your house can be obsolete.

What really sells it is that all reviewers said none of the things in video were doctored. You can really walk on the Moon or Mars. You can really build a castle on your table, and it will look real. You can finally play 2500 pts game of Warhammer 40K without bankrupcy

Minecraft IRL

By far, the most impressive demo for my money was the Minecraft demo — though Microsoft called it something like "Building Blocks" or some such, presumably so as not to fully commit to releasing a full holograph version of Minecraft. But before we could enter this virtual world — actually, the virtual entered our world — we had to strap on the development unit for the HoloLens.

It's a contraption, to be sure. There's a small, heavy block you hang around your neck which contains all the computing power. It's comprised of lenses and tiny projectors and motion sensors and speakers (or something that makes sound, anyway), and god knows what else. And then there's a screen right there in your field of view.
"It's just sitting there, perfectly flat on the table, reacting in space to your head movements"

A "screen in your field of view" is the right way to think about HoloLens, too. It's immersive, but not nearly as immersive as proper virtual reality is. You still see the real world in between the virtual objects; you can see where the magic holograph world ends and your peripheral vision begins.

But before you can apply your jaded "I've done VR before" attitude to this situation, you look down at the coffee table and there's a castle sitting right on the damn thing. It's not shimmery, but it's not quite real, either. It's just sitting there, perfectly flat on the table, reacting in space to your head movements. It's nearly as lifelike as the actual table, and there's no lag at all. The castle is there. It's simply magic.

You definitely have a big stupid grin on your face even though the contraption that's strapped to it is pressing your eyeglasses into the bridge of your nose in a painful way.

Then it's demo time. You can't touch anything, but you can look and point a little circle at objects on it by moving your head around. You learn how a "glance" is just you looking at things and pointing your reticle at them, and an "AirTap" is the equivalent of clicking your mouse. The demo involves digging Minecraft holes and blowing up Minecraft zombies with Minecraft TNT. It's basically incredible to see these digital things in real space.

You blow up a hole in the table and then you look through it to more digital objects on the floor. You blow up a hole in the wall, and tiny bats fly out. You see that behind your very normal wall is a virtual hellscape of lava and rock. You peer into the hole, around the corner, and see that dark realm extend far into space.

Skype

Microsoft’s Skype demo was as equally impressive to me as playing around with Minecraft blocks in a living room. After a two-hour keynote, Microsoft wanted me to fix a light switch. It all started by sitting down and facing some tools and a socket with exposed wiring. A little dazed and confused, I looked up and scanned across the Skype interface which was suddenly appearing in front of me, and picked a face to call. The video call popped into a little window, and my journey to fix a light switch began.

On the other end of the call was a Microsoft engineer. I could see and hear her, but she could only hear me and see exactly what I was seeing in front of me. My eyes, or the headset on my head, was relaying everything over Skype. It was a support call of sorts — here she was to help me fix a light switch. We started by pinning her little window on top of a lamp. I could then look around the room and return to the lamp to see her face. She guided me where to go. It felt strangely natural, and I didn’t need to configure anything or learn gestures other than the same "Air Tap" you use to simulate a mouse click.

While I was being talked through which real world tools we needed for the job, the Microsoft engineer called my attention to the wall with wiring and then started drawing where to position the light switch right on the wall. Thinking about it now it sounds totally surreal, but during the demo I didn’t even think about it — it just felt like I was being guided around with annotations and a helpful friend. We connected the wiring, tested it for an electrical current, and then turned the power back on and switched the light on. It was all fixed, and all by using a crazy combination of a headset, augmented reality, and Skype. It might sound gimmicky, but the applications here are truly impressive. I use YouTube guides to figure out home improvements or to service my car, but this is on another level. Imagine a surgeon performing complex surgery and writing notes in real time and guiding a colleague through it all. Imagine support calls to resolve a problem with your PC. If this works as well as Microsoft’s controlled demo, then this really has the ability to change how we communicate and learn.

Holo Studio

Microsoft's next demo didn't have us using the HoloLens prototypes directly. Instead, we watched as "Nick" (nobody in Microsoft's blue-tinted demonstration basement has last names. I asked.) manipulate objects in digital space so he could build a Koala bear or a pickup truck. It was actually quite impressive, as cameras filmed him and screens showed both Alex and the virtual objects he was manipulating in the same space in real time.

The idea was to convince us that HoloLens would unleash a wave of creators who would be able to dream up 3D objects with little to no training. It's much easier to understand what a thing is in your living room than it is in AutoCad.

Microsoft HoloLens

But sitting there after our whirlwind of actually experiencing HoloLens, my mind was elsewhere. For example, there are only a few ways to interact with this system so far:

Glance: you point your head at something.
AirTap: you make a "Number 1" sign with your hand, then move your finger down like you're depressing a lever.
Voice: you can issue commands, usually to switch what "tool" you're using.
Mouse: So actually the neatest thing is that objects you use to interact with computers can be used to interact with holograms.

That seems like enough, but it's not nearly enough. It's wildly impressive that these objects really do feel like they're out there in your living room, but it's equally depressing to know that you can't treat them like real objects.

At one point in the demo, Alex needed to put a tire on his pickup. He had to twist his body and head around to get his pointer in just the right spot and get the tire arranged just right to fix on the axle. Then, AirTap! the tire is connected. But how much easier would it be if you could grab the tire in your actual hands?

Our hands are simply more dextrous than our necks. You have finer control over small motions, you can move your hands in so many different ways and vectors, with pressure, nuance, and delicacy. Your neck and head, well, not so much.

But then Microsoft gave us 3D printed Koalas with a USB drive inside them, which was nice. And if this HoloLens thing takes off, you will be able to design your own and it will be way easier than learning current 3D design software. But not as easy as it would be if you just imagined building with holograms.

Walking on Mars

Microsoft has teamed up with NASA to let scientists explore what Curiosity sees on Mars. Instead of panoramic imagery on a computer screen, Microsoft’s demo lit up a room and turned it into Mars. I walked around the rocky terrain, bumped into the Curiosity rover, and generally just checked out a planet I will never visit in my lifetime. It’s a totally new perspective that felt like I was immersed in touring Mars, but not necessarily there. The field of view felt a little too limited to truly immerse myself and trick my brain into thinking I was really on another planet, but what impressed me most is what Microsoft has built into this experience.
I held a call with a NASA engineer, and he talked me through the terrain. I squatted to look more closely at rocks, took snapshots of various rock formations, and even planted flags for points of interest. My jaw dropped when I ventured over to a PC in the room and started to experiment with the mouse. I pulled the mouse pointer off the screen and suddenly it was on the floor next to me, allowing me to set markers in the virtual environment. It’s everything I’ve seen in demonstrations from Microsoft Research before, but here it was on my head and working.

The collaboration part was the key here, allowing me to interact with this data in a unique way, but also alongside the NASA engineer who could drop flags on the Mars terrain and guide me to look at certain sections. While this isn’t traditional productivity with a mouse and keyboard, it’s certainly something new and intriguing. I could see this type of scenario working for big teams that need to communicate across time zones and on big sets of complex data.

Overall, HoloLens is Microsoft at its most ambitious. It’s a big bet on the future of computing, the future of Windows, and ultimately the future of Microsoft itself. While the company is struggling at mobile, it wants to catch the next wave of computing and lead. Is HoloLens the next wave? Developers and consumers will be the ultimate test of that, but if anything HoloLens is an incredibly brave and impressive project from Microsoft. It’s true innovation, which is something Microsoft has lacked during its obsession with protecting Windows. It’s also another example of an experience that takes the complex technology out of the way, leaving you to experience what really matters.

[Iroscato]

Sign me the hell up. This looks to be the stuff my childhood dreams were made of
Question: Why are MS not releasing windows 9 and instead skipping straight to 10? Was 8 so harmful to create that it made them forget how to count?

[Irbis]

Tech answer I heard: because a lot of older lazily coded programs recognized Windows 9 as Windows 9x (read, 95 or 98) causing them to instantly break, plus 10 being natural marketing fit for XBox 1 / DirectX 12?

[salm]

add two screens to glasses, connect to PC, done.

And you have to get the rotational and positional head tracking right and the latency down and sychronize the the screens and get the system to run at a consistent 60 or better 75 or 90 FPS and a whole bunch of other issues. If you fail at that people will get sick and cover the planet in vomit.
This, AFAIK, is not necessary in AR because the brain has a reference point without latency which is the real world. So when seeing things in AR your body is not out of sync with your brain and therefore you don´t get sick.
It is trivial to create a crap VR system like Google Cardboard. It is difficult to create a good VR system that people actually want. There´s one that works pretty decently now but that´s it. If it was trivial we´d be swamped in VR systems.
On the other hand it is also trivial to build a crap AR system. AR has been implemented in cell phone apps and what not for years. It´s just not very good.

[Broomstick]

So.... is this going to be affordable or cost-prohibitive?

Also - I know there's a lot of hype about 3D printing and such, but producing a good design with 3D software isn't as easy as it sounds. Unless you're talking about downloading something already designed and tested, rather than creating from scratch.

[salm]

So.... is this going to be affordable or cost-prohibitive?

Also - I know there's a lot of hype about 3D printing and such, but producing a good design with 3D software isn't as easy as it sounds. Unless you're talking about downloading something already designed and tested, rather than creating from scratch.

Sure, but you don´t need any specialized skills to use AR glasses just like you don´t need specialized skills if you want to use a monitor.
Regarding the price, I doubt that it is going to be absurdly expensive. AR has been around for a long time the only problem is that nobody has managed to make it a product interesting enough for the masses at a price low enough for the masses. If it was too expensive creating it wouldn´t make much sense and I hope MS learned that from the joke that google glass was.

The prototypes of the currently only good and affordable VR goggles are selling for 350 $US.
You can order one here. Setting them up can be a bit fiddly but it isn´t really hard and it´s whole lot of fun.
https://www.oculus.com
Here and there you read how handicaped or old people profit a lot from VR/AR because it gives them a sense of mobility that they might not have, even if it is just simulated mobility.

[SpottedKitty]

It is trivial to create a crap VR system like Google Cardboard. It is difficult to create a good VR system that people actually want. There´s one that works pretty decently now but that´s it. If it was trivial we´d be swamped in VR systems.

Another potential glitch, especially if you're going for a 3D effect; not everyone can use it. This new glasses system might be different, but I've looked at the current crop of 3D TV screens, and...

<shrug> I just plain don't see it. There's no jumping-out-at-me 3D effect like I see with a simple stereoscope. The various cross-your-eyes-and-get-a-headache tricks don't work with me, either. Depending on how common this is, that's going to limit the potential user base.

[Irbis]

So.... is this going to be affordable or cost-prohibitive?

It should not cost much more than two good tablets, or say new Xbox with Kinect, IMHO.

Also - I know there's a lot of hype about 3D printing and such, but producing a good design with 3D software isn't as easy as it sounds.

The thing is, our hands are much more dexterous than mouse. It really shouldn't be that hard to produce interface that easily beats 2D screen - look at multitouch gestures, they beat the mouse in functionality despite being 2D too.

Here and there you read how handicaped or old people profit a lot from VR/AR because it gives them a sense of mobility that they might not have, even if it is just simulated mobility.

It's better, other Microsoft prototype is AR system for the blind. And it apparently works great too

[salm]

Another potential glitch, especially if you're going for a 3D effect; not everyone can use it. This new glasses system might be different, but I've looked at the current crop of 3D TV screens, and...

<shrug> I just plain don't see it. There's no jumping-out-at-me 3D effect like I see with a simple stereoscope. The various cross-your-eyes-and-get-a-headache tricks don't work with me, either. Depending on how common this is, that's going to limit the potential user base.

You don´t matter.

The amount of people not being able to see in 3D is insignificant. Furthermore the 3D effect isn´t necessary for VR or AR to be useful. People who can´t see stereoscopic, for example people with only one eye manage fine in the real world. The VR/AR world is just going to be the same for them.

[Broomstick]

The amount of people not being able to see in 3D is insignificant.

Incorrect.

With current 3D systems for movies and TV's about 1 person in 6 either has trouble using them, or can't use them with serious problems like nausea. That's a significant number.

(If you don't like long posts you can skip the rest, it's just expanding on the above statement)

The one-eyed actually have it easy - while they won't get the 3D effect and will just get the view they normally do they shouldn't be troubled by nausea and double-vision that some of the two-eyed-but-issues-with-3D people have. Nor is this always a mild problem - I know of at least one person for whom modern 3D image systems are such a problem she literally pukes.

There are work-arounds, of course - for instance, the 3D system we have on our TV at home has a setting on the glasses where it removes one of the two images so that the picture is 2D for those who can't tolerate the 3D effect, allowing everyone to watch the same movie at the same time. (They also have glasses that actually fit over my prescription glasses, which isn't always the case either. Fuck you, too, Google Glass).

Furthermore the 3D effect isn´t necessary for VR or AR to be useful. People who can´t see stereoscopic, for example people with only one eye manage fine in the real world. The VR/AR world is just going to be the same for them.

There is more to 3D perception than just stereoscopic vision, which is why one-eyed people are able to drive cars and fly airplanes without crashing into stuff all the time.

One advantage to actual holograms is that they allow the person to move around the object, getting different views of it, which the brain can also use to construct a 3D internal image because seeing 3D is NOT just about having two eyes! There's a crapload of information processing that occurs in the brain to meld those two different viewpoints into a 3D view (when that mechanism isn't working you can have someone with two fully functional eyes who nonetheless can't perceive 3D), along with other cues such as relative sizes, "blueing" with distance, changes in image as an object is rotated or the viewpoint moves around it... Some people complain that modern 3D movies appear "flat" or as if there are flat cut-outs of objects at different distances and what probably means is that their brains are not getting cues they get in the real world and thus the perception that something isn't quite right here.

I actually get the opposite effect of a lot of people - one of my vision problem is that I don't have much depth perception in real life (I almost had to take the one-eyed pilot vision test, it's that bad). I compensate quite well for it but any vision test designed to catch problems with stereoscopic vision is going to show I have a problem with it. However - with modern movie and TV 3D I actually get MORE depth perception than I do in ordinary life. I don't quite understand how that happens. It's not as wonderful as that might sound, though. Sure, it's cool actually seeing depth in a way I don't normally but my brain has lived a half century without that input. It is very, very tiring for me to watch 3D images of that sort, and I suspect it's because my brain is having to process stuff it normally doesn't, or process in a manner it normally doesn't. With time it is becoming less exhausting (because even old brains retain some ability to adopt) but it's different than what vision normally is for me.

The normally sighted don't quite get that there is a real issue here, and it's not just a teeny minority of people. "Normal" vision is an average, and one I suspect was arrived at before modern statistical research. Hence why more and more people seem to be getting "corrective lenses"

It's a bit like the issue with color "blindness" - it's a more complicated situation than it first appears, and it's not as binary as you might think. Computer programmers are getting more savy, and there are more resources out there to help designers avoid problems, but it's maddening to encounter a website where you simply can not read the text, or a video game where important information is lost against a background (thank you, Blizzard, for giving us a fix for that problem in WoW, the Barrens and Stranglethorn Vale are a LOT easier to wander through now) and have people just shrug, say you're weird, dismiss the problem as too few people to give a fuck about, and so forth.

[Lagmonster]

The only relevant point from Microsoft's point of view is that, even if it is true that one in six people can't comfortably use the device, that still leaves a satisfactorily huge pool of potential customers.

[Broomstick]

Microsoft actually does a reasonably decent job of making their stuff accessible. Sure, 5/6 of the population is the bulk of potential customers but given the sheer number of people, 1/6 of the population still runs into the tens of millions.

[salm]

The amount of people not being able to see in 3D is insignificant.

Incorrect.

With current 3D systems for movies and TV's about 1 person in 6 either has trouble using them, or can't use them with serious problems like nausea. That's a significant number.

That lumps together several different problems. Not being able to see a 3D effect is completely different from seeing it but getting sick.
People getting sick is a really big problem in VR, a lot worse than on 3D TVs. The vast majority of people get sick in VR when confronted with the wrong type of input. Software makers have to be very careful what they present to the user. That is the reason why you can not just take any normal computer game, make it stereoscopic and send it to a VR device. Some games work pretty good by chance. Others only have to be slightly modified and again others have to be modified so much that not much of the original game is left.
Personally I don´t get motion sick very easily. I can read in a car, I don´t get seasick and have never experienced the slightest bit of nausea in front of a TV or Computer screen. However present me the wrong footage in VR for a second or two and I´m sick. This sickness used to last for half an hour in the beginning. Now, after regularily going into VR for half a year the sickness goes away very quickly after the event triggering the sickness which is usually the view doing certain kinds of other motions than my head.

However, this is different from not being able to see stereoscopic. Most people have stereoscopic vision (over 97% acording to wikipedia). Furthermore it doesn´t even matter if you have stereoscopic vision. VR is just as good without. A while ago I tested that on my father. He lacks sterescopic vision and I sent him diving with some VR sharks. He enjoyed it just like everybody else and was surprised when he was finally eaten by the Megalodon just like anybody else. And I assume the same is true for AR.

[Vendetta]

The amount of people not being able to see in 3D is insignificant.

Incorrect.

With current 3D systems for movies and TV's about 1 person in 6 either has trouble using them, or can't use them with serious problems like nausea. That's a significant number.

That lumps together several different problems. Not being able to see a 3D effect is completely different from seeing it but getting sick.

Both are a lost sale if your business is selling 3D.

So no, not really.

[salm]

Incorrect.

With current 3D systems for movies and TV's about 1 person in 6 either has trouble using them, or can't use them with serious problems like nausea. That's a significant number.

That lumps together several different problems. Not being able to see a 3D effect is completely different from seeing it but getting sick.

Both are a lost sale if your business is selling 3D.

So no, not really.

Neither is lost sale. People without stereöscopic vision benefit in the same way from VR/AR as other people and the motion sickness can be eliminated.
They are different, though, in the sense that one thing is a non solvable handicap on the persons side and the other is a solvable handicap on the devices side.

So, have you been able to try a Rift yet since our last conversations?