MIT: The Internet is now a 6th human sense

[rhoenix]

(Quite honestly, I wasn't sure whether to post this here, or in G&C - I posted it here in SLAM due to the possible sociological impacts from such technology.)

LONG BEACH, California -- Students at the MIT Media Lab have developed a wearable computing system that turns any surface into an interactive display screen. The wearer can summon virtual gadgets and internet data at will, then dispel them like smoke when they're done.

Pattie Maes of the lab's Fluid Interfaces group said the research is aimed at creating a new digital "sixth sense" for humans.

In the tactile world, we use our five senses to take in information about our environment and respond to it, Maes explained. But a lot of the information that helps us understand and respond to the world doesn't come from these senses. Instead, it comes from computers and the internet. Maes' goal is to harness computers to feed us information in an organic fashion, like our existing senses.

The prototype was built from an ordinary webcam and a battery-powered 3M projector, with an attached mirror -- all connected to an internet-enabled mobile phone. The setup, which costs less than $350, allows the user to project information from the phone onto any surface -- walls, the body of another person or even your hand.

Maes showed a video of her student Pranav Mistry who she describes as the brains behind the project. Mistry wore the device on a lanyard around his neck, and colored Magic Marker caps on four fingers (red, blue, green and yellow) helped the camera distinguish the four fingers and recognize his hand gestures with software that Mistry created.

The gestures can be as simple as using his fingers and thumbs to create a picture frame that tells the camera to snap a photo, which is saved to his mobile phone. When he gets back to an office, he projects the images onto a wall and begins to size them.

When he encounters someone at a party, the system projects a cloud of words on the person's body to provide more information about him -- his blog URL, the name of his company, his likes and interests. "This is a more controversial [feature]," Maes said over the audience's laughter.

In another frame, Mistry picks up a boarding pass while he's sitting in a car. He projects the current status of his flight and gate number he's retrieved from the flight-status page of the airline onto the card.

"If you need to know what time it is, it's as simple as drawing a watch on your arm," Maes said, while Mistry used his right finger to draw a circle on his left wrist. The face of a watch popped up on his hand, which the audience liked.

When Mistry folds his hands in "namaste" fashion, the system opens a menu to allow him to choose an application. If he wants to read e-mail on his phone, he draws an @ symbol in the air with his finger. He can project a phone pad onto his palm and dial a number without removing the phone from his pocket. As he reads the newspaper on the subway he can project a video onto the page that provides more information about the topic he's reading.

Maes and Mistry told Wired they've been working on the project for four months, day and night, and have filed a patent for it.

Maes' MIT group, which includes seven graduate students, were thinking about how a person could be more integrated into the world around them and access information without having to do something like take out a phone. They initially produced a wristband that would read an RFID tag to know, for example, which book a user is holding in a store.

They also had a ring that used infrared to communicate by beacon to supermarket smart shelves to give you information about products. As you grab a package of macaroni, the ring would glow red or green to tell you if the product was organic or free of peanut traces -- whatever criteria you program into the system.

"We wanted to make information more useful to people in real time with minimal effort in a way that doesn't require any behavior changes," Maes said. "The wristband was getting close, but you still had to take out your cell phone to look at the information."

That's when they struck on the idea of accessing information from the internet and projecting it. So someone wearing the wristband could pick up a paperback in the bookstore and immediately call up reviews about the book, projecting them onto a surface in the store or doing a keyword search through the book by accessing digitized pages on Amazon or Google books.

They started with a larger projector that was mounted on a helmet. But that proved cumbersome if someone was projecting data onto a wall then turned to speak to friend -- the data would project on the friend's face. Last month, they switched to a smaller projector and created the pendant prototype to be worn around the neck.

The TED demo was the first time they've shown it in public, though they're far from making a commercial product or forming a company around their invention. "But we're really excited about the potential," Maes said.

They learned recently that cellphone makers soon plan to release cellphones with projectors integrated in them, which will simplify their system even more.

Not only does this make me smile to know MIT was behind this, I think this idea is an excellent one - however, some of the effects of such technology might have interesting effects sociologically. I speak specifically about being able to see someone's website URL and IM contact info popup when you look at someone's face, which could be interesting.

[Duckie]

If integrated with a helmet or visor as a display on the visor's screen overlayed over what the visor is seeing (thus being your vision without the visor + everything the visor is displaying) instead of a physical projection- instant augmented reality occurs. Cool stuff.

[Formless]

If integrated with a helmet or visor as a display on the visor's screen overlayed over what the visor is seeing (thus being your vision without the visor + everything the visor is displaying) instead of a physical projection- instant augmented reality occurs. Cool stuff.

Or sunglasses. Never forget sunglasses! You would have to dink around with the interface to make it work, however. In exchange, no one knows what information you are looking at, which could be a good way to solving any social issues that crop up from that "I see your face and project your blog on your shirt" function.

Hell, I had a similar idea for a sci-fi character I was writing-but-never-finished. But then, he was a cyborg as well, so...

The social ramifications of this are hard to know. If it is an extension of cell phone technology, it could just as easily turn into a fad as become a component of the Technological Singularity. What would it cost? If it is too expensive, it could remain the exclusive domain of rich businessmen for an extended period of time, much like blackberries were until recently. Not everyone is going to care about being able to wear it as long as they can use their phone just fine. Call me a ludite, but from that standpoint, that is all I really need from a phone. Everything else is just window dressing. Granted, really cool window dressing, but nothing I can't do without.

[rhoenix]

Or sunglasses. Never forget sunglasses! You would have to dink around with the interface to make it work, however. In exchange, no one knows what information you are looking at, which could be a good way to solving any social issues that crop up from that "I see your face and project your blog on your shirt" function.

Quite true - I would fully expect to see special sunglasses or glasses for this sort of technology, should it take off commercially (and assuming we're past this whole "worst recession in 35 years" thing.)

The social ramifications of this are hard to know. If it is an extension of cell phone technology, it could just as easily turn into a fad as become a component of the Technological Singularity. What would it cost? If it is too expensive, it could remain the exclusive domain of rich businessmen for an extended period of time, much like blackberries were until recently. Not everyone is going to care about being able to wear it as long as they can use their phone just fine. Call me a ludite, but from that standpoint, that is all I really need from a phone. Everything else is just window dressing. Granted, really cool window dressing, but nothing I can't do without.

Well, like with most technology we almost consider ubiquitous today, it'll likely stay expensive for a short time while the design is made more commercially efficient, and then the costs will likely come down. I didn't expect people's use of cell phones to overtake people's use of landline phones, but that had to do with (relative) ease of purchase, and convenience of use. Should this happen the same way...well, just like a decade ago when you saw someone talking to nobody nearby you thought they're just crazy (and not on a cell phone), soon we might see people drawing symbols in midair like they were LARP geeks, and then even that might become "normal."

Then again, this is my opinion based on what I've seen - consult your local exchange rate for worth.

[Mayabird]

No, my sixth sense is sensing changes in air pressure. It comes in the form of splitting headaches whenever the changes are occurring rapidly (I always knew when storms were coming; it wasn't very useful) and general discomfort when slower. This would be number seven, even though I was hoping sensing Earth's magnetic field (as many animals can do) would be seven, making this eight.

[Sriad]

No, my sixth sense is sensing changes in air pressure. It comes in the form of splitting headaches whenever the changes are occurring rapidly (I always knew when storms were coming; it wasn't very useful) and general discomfort when slower. This would be number seven, even though I was hoping sensing Earth's magnetic field (as many animals can do) would be seven, making this eight.

Ninth; the kinesthetic sense (or Proprioception if you prefer) is the awareness of where the parts of your body are relative to each-other. For example, disruption of this sense is what makes the field-sobriety test of touching your nose with your eyes closed indicative of intoxication.

[Oskuro]

No, my sixth sense is sensing changes in air pressure. It comes in the form of splitting headaches whenever the changes are occurring rapidly (I always knew when storms were coming; it wasn't very useful) and general discomfort when slower.

Ah, so I'm not alone in my barometric weirdness!

Also, will we get to see health bars on top of people?

[RogueIce]

I bet if/when the police start using the things to, say, see who has an active warrant just by looking at you, or find out your name, driver's license info, criminal record, driving record, etc with just a turn of their head and finger movements (basically stuff they could find now if they have your driver's license in hand and their radio and/or vehicle computer) people will scream bloody murder.

EDIT: One presumes if they do this, it'll at least be with some headset so only they see it, not "Hey, that person has a murder warrant!" popping up on your t-shirt or nearby wall.

[Ryan Thunder]

I bet if/when the police start using the things to, say, see who has an active warrant just by looking at you, or find out your name, driver's license info, criminal record, driving record, etc with just a turn of their head and finger movements (basically stuff they could find now if they have your driver's license in hand and their radio and/or vehicle computer) people will scream bloody murder.

EDIT: One presumes if they do this, it'll at least be with some headset so only they see it, not "Hey, that person has a murder warrant!" popping up on your t-shirt or nearby wall.

Probably wouldn't be that good. I imagine they'd have to get your basic information and then search it. They just wouldn't have to go back to the squad car to do it.

[sketerpot]

No, my sixth sense is sensing changes in air pressure. It comes in the form of splitting headaches whenever the changes are occurring rapidly (I always knew when storms were coming; it wasn't very useful) and general discomfort when slower. This would be number seven, even though I was hoping sensing Earth's magnetic field (as many animals can do) would be seven, making this eight.

Ninth; the kinesthetic sense (or Proprioception if you prefer) is the awareness of where the parts of your body are relative to each-other. For example, disruption of this sense is what makes the field-sobriety test of touching your nose with your eyes closed indicative of intoxication.

Tenth. Don't forget the vestibular system of your inner ear, which detects linear and rotational acceleration. That includes acceleration due to gravity, so it also tells you which way is down.

[PeZook]

I bet if/when the police start using the things to, say, see who has an active warrant just by looking at you, or find out your name, driver's license info, criminal record, driving record, etc with just a turn of their head and finger movements (basically stuff they could find now if they have your driver's license in hand and their radio and/or vehicle computer) people will scream bloody murder.

They probably would, but I actually expected something like this to happen with the miniaturization of computer technology. I mean, fort years ago cops couldn't dream of having computer terminals in their squad cars, now every car has one. It's another logical step to make the terminals portable and equip every single cop with them. A visor is just an accessory.