David McCoy

Back in March 2008, Jackie Fenn and I interviewed Dr. Thad Starner, a true pioneer in wearable computing.  Here’s the link to the interview we host on Gartner.com.  Note, you don’t have to be a paying client to read our “Gartner Fellows Interviews.”  We fellows make these available as part of our Brand-Culture-Ideas-People initiative.

I met with Thad last week, and he gave me an update on one of the most “out there” aspects of the interview – being able to read imagined sign language through the use of Functional Magnetic Resonance Imaging (FMRI). It turns out that you can do it – read imagined sign language.  This new discovery was only an idea at the time of the interview.

In the original interview, Thad said:

The new crazy thing is that, since sign language is actually distributed throughout the motor cortex, much more than say speech is with the larynx and the throat and tongue muscles, we might literally be able to pull off sign language from the brain, which would be astonishing if we can do it…

The movement for the larynx and your tongue and your jaw is in a very small section of the brain. The movement for your shoulders, elbows, and fingers is scattered much more throughout the motor cortex. And we think we can actually pull out signed phrases through FMRI (functional magnetic resonance imaging).

I asked for clarity:

To clarify, you would not actually have to go through the physical sign language movements. You could just think about the specific movements, and that would be enough to trigger the motor cortex?

Thad responded in the positive:

It turns out that imagined movements also activate those regions… How much can we distinguish? Currently, we’re going to try our first experiments just on pairs of signs that are maximally different, like cold versus hot, because the motion should be very distinctive in the motor cortex. If that works, we’re going to go to phrases, and – this is all in FMRI machines – if that works, then we’ll start going to mobile interfaces like functional near-infrared imaging.

Well, I can tell you that they actually have been able to read imagined sign language from the brain.  While the full details are yet to be released – this is really hot information, fresh from the lab – the researchers have been able to distinguish between certain “maximally different” signs. 

I understand that Thad spent about 3 hours inside an FMRI machine during the test.  These researchers are serious about their craft.  More on this will be forthcoming as they publish the results, but I wanted to give you an quick update on this fascinating area of research.