Kinect 2 will be so powerful it will be able to lip read!

The Kinect 2, besides having the ability to “see” much closer (20-40 cm!) than V1 will be so powerful it will (rumor has it) enable your program of choice (albeit using Windows 7/8) to lip read, detect when people are angry (or otherwise), and determine in which direction they are facing.

http://www.eurogamer.net/articles/2011-11-25-kinect-2-so-accurate-it-can-lip-read

Having gotten into chat bots when I was looking for a C# “chat” library for my “talking head”  animatronic projects (I have since pretty much been reduced to using an avatar-based system since servos are just too damn noisy…), this looks to be really cool development for a natural spoken word input (as opposed to using a keyboard, or fairly low res Windows voice recognition) chat bot interface. I have more recently used the Kinect V1 for person tracking and it already works better than any other sensor (sonar, IR, etc.) in the price range (<$100), but full on lip reading would be quite remarkable indeed!

Should be a real boon for anyone making chatting robots that interact with real people in RL (like a kiosk bot, as opposed to on a web page over the internets).

Animatronics projects:
Elvis
Elvis- technical1
Elvis- technical2
Chimp

Cool article! Thanks for the link, Carl.

Wow!
One more step for natural conversation man/machine!

I was talking to a biometrics guru today. He told me that iris recognition has progressed that far that people can be recognised from 40 meters (no typo!!!!) distance. Wouldn’t that be an awesome addition as well for entertainment parcs, stores, maills etcetera?

Erwin Van Lun - Nov 30, 2011:

I was talking to a biometrics guru today. He told me that iris recognition has progressed that far that people can be recognised from 40 meters (no typo!!!!) distance. Wouldn’t that be an awesome addition as well for entertainment parcs, stores, maills etcetera?

The ability of machines to ID people is getting to the point where very interesting applications are bound to start appearing- I think one of the more frustrating parts of human-machine interactions is the inability of machines to passively ID people (or even recognize when a person is near, let alone who that person is).  We still rely almost completely on the keyboard “log-in” to do the seemingly trivial task (for people at least) of recognizing human presence, let alone uniquely identifying someone (“Greetings generic blob*” VS “Hi Erwin!”).

*Blob tracking is one of the crudest, and most widely used, technique for object recognition in computer vision.

he was also talking about an iPhone app that identifies humans and blocks transactions when the owner has not be seen or heard for a while….

Carl, I’ve seen your animatronics and they are great.
About keeping contact robot/human I just remembered Hansons Robotics works. I don’t know the real point that thir affirmations can be considered true, but they say that their AI can allow their Animatronics to keep direct eye contact with interlocutor.
Also I’d like to share with you an article published in http://blogs.scientificamerican.com/guest-blog/2011/09/15/robots-and-new-technologies-programmed-to-understand-and-interact-keep-it-future-friendly/ about robot/human natural interaction

Fatima Pereira - Dec 8, 2011:

Carl, I’ve seen your animatronics and they are great.

Thanks Fatima

Fatima Pereira - Dec 8, 2011:

About keeping contact robot/human I just remembered Hansons Robotics works. I don’t know the real point that thir affirmations can be considered true, but they say that their AI can allow their Animatronics to keep direct eye contact with interlocutor…

Thanks for the link to article, interesting stuff for sure- it has long been the purview of very expensive sensors and sophisticated programming to get robots to see and be able to respond to humans let alone their gestures, eye movements, etc.  That is one of the reasons the Kinect is so cool- it is a very sophisticated yet affordable sensor, has (better all the time) SDK support from the maker (MS) and can already recognize and track up to two people (including gestures via ‘skeletal tracking’!) in it’s field of vision- very useful for creating a chat bot that can give the appearance of physical environmental (as opposed to simple command line and/or speech/audio input) awareness.