The Volokh Conspiracy

"Visual Image Reconstruction from Human Brain Activity":

As best I can tell, this article in Neuron reports that an MRI can be used to determine what the subject is currently seeing, based on several hundred past MRIs done when the subject saw different random images. Pink TTentacle reports further:

The scientists were able to reconstruct various images viewed by a person by analyzing changes in their cerebral blood flow. Using a functional magnetic resonance imaging (fMRI) machine, the researchers first mapped the blood flow changes that occurred in the cerebral visual cortex as subjects viewed various images held in front of their eyes. Subjects were shown 400 random 10 x 10 pixel black-and-white images for a period of 12 seconds each. While the fMRI machine monitored the changes in brain activity, a computer crunched the data and learned to associate the various changes in brain activity with the different image designs.

Then, when the test subjects were shown a completely new set of images, such as the letters N-E-U-R-O-N, the system was able to reconstruct and display what the test subjects were viewing based solely on their brain activity.

Possible applications:

According to the researchers, further development of the technology may soon make it possible to view other people’s dreams while they sleep....

The researchers suggest a future version of this technology could be applied in the fields of art and design -- particularly if it becomes possible to quickly and accurately access images existing inside an artist’s head. The technology might also lead to new treatments for conditions such as psychiatric disorders involving hallucinations, by providing doctors a direct window into the mind of the patient....

Cool and scary. Thanks to my friend Haym Hirsh for the pointer, and to David Byrne for the mental soundtrack.

OrinKerr:
Actually, my mental soundtrack for that one was the Romantics. Doesn't really make sense, but that's brains and 80s music for you.
12.12.2008 4:25pm
geokstr:
Great. Just what governments need - a way to tell what you're thinking. Can "thought crimes" for political incorrectness be very far off?
12.12.2008 4:26pm
ShelbyC:

an MRI can be used to determine what the subject is currently seeing



Can't you just look where they're looking?
12.12.2008 4:26pm
Portland (mail):
An MRI scanner is a huge, bulky machine which requires the subject to hold perfectly still. The resolution they are getting is very crude (no pictures like we understand pictures.) Government thought-reading is some ways off.

The first applications of this technology are likely to be in research and in enabling people with neuromuscular disorders (ALS, spinal trauma, etc.) to communicate with bionics.
12.12.2008 4:56pm
ARCraig (mail):
Pretty much any technology can potentially be abused by the government (though not this one in its current state). That's no reason to be a Luddite. I hate speed revenue camera traps as much as your next radical libertarian, but that doesn't mean I think cameras never should have been invented.

This is cool, and as the article notes has the real potential to help people with mental problems. And instantly transforming a mental image into a real one has been the Holy Grail of visual arts since our ancestors were scribbling on the cave walls at Lascaux.
12.12.2008 5:13pm
geokstr:
Portland (mail):

Government thought-reading is some ways off.

Don't be so certain about how far off it is:
Moore's Law

At the speed that increases (or is that "decreases"?) in miniaturization are going, I'd say it's a decade off at most, just in time for Chelsea Clinton's inauguration.

You must be a young-un. I can remember when the first company I worked for, a huge one named Northwestern Mutual Life, had an entire floor devoted to a mainframe computer that does about what your average off-brand laptop does now, only less reliably, and you needed a PhD in Computer Science just to know where to plug it in and how to turn it on.

Used to be, your "portable" phone came with wheels and a pull handle, and all you could do is make phone calls. Wrist watch TVs have been around for decades already.

And once the military (of any country) learns that you can see into someone's thoughts without having to duck their heads under water at all, you can bet there will be a top secret well-funded program specifically designed to shrink the size and increase the capabilities of such a device.
12.12.2008 5:18pm
Thomas_Holsinger:
I shudder to think what some researchers would make of this if a subject, instead of watching the letters "N-E-U-R-O-N" were instead to be watching the letters:

M -- O -- R -- O -- N
12.12.2008 5:18pm
Sarcastro (www):
Technology gives government too much power, and should be banned from the government just like religeon.

"ceremonial engineering" should be pretty cool. I see it as using a lot of those zappy orbs.
12.12.2008 5:20pm
geokstr:
But don't get me wrong. I love the scientific possibilities of this and any other technological breakthroughs, but remember, even before electricity, the founders of this country had learned not to trust those in power.
12.12.2008 5:21pm
MCM (mail):
An MRI scanner is a huge, bulky machine which requires the subject to hold perfectly still. The resolution they are getting is very crude (no pictures like we understand pictures.) Government thought-reading is some ways off.


What geokstr said. Technology improves very quickly once initial breakthroughs like this one are made.
12.12.2008 5:29pm
ARCraig (mail):

But don't get me wrong. I love the scientific possibilities of this and any other technological breakthroughs, but remember, even before electricity, the founders of this country had learned not to trust those in power.


And many of them were also avid technophiles. Distrust the government, not scientific progress just because it might be used by the government someday.
12.12.2008 5:40pm
HeScreams:
Funny you should post this -- just yesterday I was talking to a person who does similar research. See this article in "Nature".

@geokstr, MCM: Problems like this aren't just a matter of computing power or circuit miniaturization (which are the core of the examples geokstr cited). This kind of technology relies on machine learning algorithms; and algorithm development doesn't follow anything like Moore's law. Having worked in the field myself, I'm always quite skeptical of anything with an "Artificial Intelligence" flavor to it. Impressive sounding claims like this usually turn out to be much more limited when you get into the nuts and bolts.

IMHO, brain reading is still a long, long way off.

If you're unconvinced, think about NetFlix. It's supposed to have this wonderful algorithm that predicts what movies you'll like based on what movies other people like, right? But if you use NetFlix, you know that all it does is recommend films in the same genre, or with the same actor. Same with Amazon.com's prediction algs. The same kind of pattern learning algs are what's used in these 'brain readers', so don't buy your tin-foil hat just yet...
12.12.2008 5:51pm
ewannama:
Quite a huge leap from recognizing/connecting one person's brain pattern based on one image previously viewed many times by that person, to reading other people's thoughts. We've all wired our brains differently within pretty wide bounds. It's a leap even to connect all the variables within a single image for a singe person--will a vision of a new house create a close enough pattern to point to the cartoon card of a blue house with white-picket fence?
12.12.2008 6:09pm
ewannama:
Skepticism aside, there are a lot of neat possibilities in the realm of Firefox...
12.12.2008 6:10pm
HeScreams:
@ ewannama@ : Firefox technology is actually much further along than brain-image reading technology. See the SciAm article.

Now this I expect to see in my lifetime... at least for artificial limbs, if not Russian superjets.
12.12.2008 6:52pm
Fred Barth (mail):
The article says: "A spokesman at ATR Computational Neuroscience Laboratories said: "It was the first time in the world that it was possible to visualise what people see directly from the brain activity."

It has been possible since the late 90s to assess whether people were experiencing an image falling with a very limited set of categories based on neuroimaging data. Brute force processing can extend the method. "Changes in brain activity" is imprecise, and what fMRIs actually measure is open to question. A given image of a kind of activity in a given region can represent a complex range of causes and interactions. Based on how this was reported, it's not clear what was really said, what researchers thought they had accomplished, and what's really new.
12.12.2008 6:55pm
fortyninerdweet (mail):
Are you sure this post wasn't taken from a recent episode of "Fringe"? Sure sounds like the same fringe science issue.
12.12.2008 7:02pm
John Moore (www):
There is a more interesting (scary) use of this technology.

The MRI can be more easily used to tell if a person previously saw an image. In fact, other neuro sensors can do this (perhaps not as well).

This becomes a potentially accurate lie detector for certain assertions, and detecting other familiar patterns could be used to detect those familiar with "incorrect" ideas.

Or, more prosaic, perhaps to tell who has seen a lot of child porn.
12.12.2008 7:04pm
MCM (mail):
Are you sure this post wasn't taken from a recent episode of "Fringe"? Sure sounds like the same fringe science issue.


It's published in a peer-reviewed neuroscience journal... not sure what you're looking for here...
12.12.2008 7:58pm
Bob Goodman (mail) (www):
I'm surprised progress in this field has been so slow. I remember a garage-style inventor (very good with remote controls) in Ill. who ~30 years ago claimed some success in controlling machines using just surface potentials (scalp electrodes as in EEG) as input, using pattern recognition. Thought it had obvious applications in spinal injuries, he hyped it as a step toward mind reading.

Meanwhile work on nonmaterial methods of mind reading have suffered from association with charlatans and even (thru trivializ'n) from ass'n with entertainers. George P. Hansen has noted that the entire field of "paranormal" research is underfunded and given little respect.
12.12.2008 8:15pm
Elmer:
Looking at blood flow distribution to determine visual patterns is like mapping a car's fuel use over time to determine where it has been driven. These results are impressive, but resolution limits, statistics, and the need to calibrate for the individual mean that this window on the mind will always have dirty glass. No matter how well blood flow can be measured, is it possible that researchers could have looked at the flows and concluded that I spent one night thinking I was the new Fed chair, and a zombie? That I controlled the money supply by my rate of brain eating? That I got confused, initially believing that more brains reduced money supply, but later believing the opposite, and afraid that I'd caused instability?
12.12.2008 9:00pm
John Burgess (mail) (www):
Anyone want to join me in patenting the system by which the hallucinations of schizophrenics is rendered in 2- and 3-dimensional art?

I'm also patenting the art rendered during wet dreams because while art doesn't pay, porn does!
12.12.2008 9:17pm
David Warner:
Seems like thoughts which would be controversial to governments are the abstract ones, hence difficult to render as mere images.
12.12.2008 9:43pm
John Moore (www):
@Bob Goodman

That technology used patterns of electroencephalograhic signals. It was quite crude. Work with biofeedback in the late '60s/early '70s demonstrated that people could learn to control measurable aspects of brain waves.
12.12.2008 9:56pm
Portland (mail):

Don't be so certain about how far off it is:
Moore's Law


Moore's law is not applicable to the performance of MRI machines, which are large and bulky because of the nature of the emitters and the detectors, not microchips.

Even Moore's law has produced only a rapid improvement in processing power; functionality (what we're discussing here) has been much more gradual in its progress. The chip running it is much faster, but my word processing program still takes the same time to load it did in the 90's, and is not notably more useful.

Moore's law is not representative of the normal pattern of technological development in general. MRI machines are not likely to become much smaller; even if they did, that would not change the requirement that the subject remain perfectly still in a predetermined location between the emitter and the detector. Getting around that is going to be very tricky.

Some technologies, like computer chips, improve rapidly, and some, like batteries, or handguns, are difficult to improve substantially. Eventually they will be replaced, but it isn't likely to be soon.
12.12.2008 11:05pm
ewannama:
Oops, I misread the article. Anyway, thanks "HeScreams" for the link.
12.13.2008 4:51pm
John Moore (www):
As a software engineer (named Moore coincidentally), I have observed that, at least until the internet, progress in hardware was dramatically faster than progress in software.

For example, computer scientists in the 1950's had great expectations for Artificial Intelligence. They were no more accurate than the contemporaneous predictions of personal commuting by flying car.

Moore's law doesn't apply to everything.
12.13.2008 7:21pm
Anatid:
See a similar article.

Don't forget an important part of this article - each person has a unique, and completely different, activation pattern when viewing a particular image. The only way to identify a viewed image from an fMRI (with the current technology) is to already have an averaged fMRI made from several prior exposures to the exact same image. Similar images, or images of the same object from different angles, don't help.
12.15.2008 8:23pm
Neuro:
Also, for those worried about "mind reading", note that the area they examined -visual cortex- has special topographic properties that make this possible. Nothing to do with the analysis, the software, the methods; the brain just makes this possible in some areas but not others.

The things you see are mapped onto your visual cortex, in a way that preserves the spatial patterns: this has been known for years, though previously you had to do things like sacrifice and cell-label an animal in mid-viewing to observe the effect.

The visual cortex activates in ways that preserve the shape of the word "neuron" in a recognizable way. But the effect doesn't extend beyond vision: these tools can't see the concept of "neuron" anywhere in the brain.

Methods to accomplish that are either categorically impossible, or so far off that neuroscientists can't imagine how it might be possible.
12.17.2008 1:21pm

Post as: [Register] [Log In]

Account:
Password:
Remember info?

If you have a comment about spelling, typos, or format errors, please e-mail the poster directly rather than posting a comment.

Comment Policy: We reserve the right to edit or delete comments, and in extreme cases to ban commenters, at our discretion. Comments must be relevant and civil (and, especially, free of name-calling). We think of comment threads like dinner parties at our homes. If you make the party unpleasant for us or for others, we'd rather you went elsewhere. We're happy to see a wide range of viewpoints, but we want all of them to be expressed as politely as possible.

We realize that such a comment policy can never be evenly enforced, because we can't possibly monitor every comment equally well. Hundreds of comments are posted every day here, and we don't read them all. Those we read, we read with different degrees of attention, and in different moods. We try to be fair, but we make no promises.

And remember, it's a big Internet. If you think we were mistaken in removing your post (or, in extreme cases, in removing you) -- or if you prefer a more free-for-all approach -- there are surely plenty of ways you can still get your views out.