On the road to issuing silent commands to machines

“Fingers!”, Gerwin Schalk sputtered, waving his hands in the air. “Fingers are made to pick up a hammer.” He prodded the table, mimicking the way we poke at computer keyboards. “It’s totally ridiculous,” he said.

I was visiting Schalk, a 40-year-old computer engineer, at his bunkerlike office in the Wads­worth Center, a public-health lab outside Albany that handles many of New York State’s rabies tests. It so happens that his lab is also pioneering a new way to control our computers — with thoughts instead of fingers. Schalk studies people at the Albany Medical Center who have become, not by choice, some of the world’s first cyborgs.

One volunteer was a young man in his 20s who suffers from a severe form of epilepsy. He had been outfitted with a temporary device, a postcard-size patch of electrodes that sits on the brain’s cortex, known as an electrocorticographic (ECoG) implant. Surgeons use these implants to home in on the damaged tissue that causes seizures.

Schalk took advantage of the implant to see if the patient could control the actions in a video game called Galaga using only his thoughts. In the videotape of this experiment, you see a young man wearing a turban of bandages with wires running from his head to a computer in a cart. “Pew, pew,” the ship on the computer screen whines, as it decimates buglike creatures. The patient flicks the spaceship back and forth by imagining that he is moving his tongue. This creates a pulse in his brain that travels through the wires into a computer. Thus, a thought becomes a software command.

On the day I stopped by his office, Schalk hit a button on his computer, and Pink Floyd blasted from his speakers. He was running an experiment to see what happens to people’s brains when they listen to Another Brick in the Wall, Part 1 (a question that has occurred to any stoner who ever contemplated human consciousness in the glow of stereo lights). Weeks before, Schalk played the Pink Floyd song for some of his epileptic volunteers and recorded the activity in the parts of the brain that process sound. Schalk showed me a volume meter on his computer screen — this was a brain, tracking the roar of a guitar solo. It worked just like any other volume metre, but in one experiment, Schalk found the brain did something unexpected. When he interrupted the Pink Floyd song with moments of silence, the brain’s volume metre continued to tremble up and down, as if the song were still playing. This, Schalk said, showed the brain creates a model of what it expects to hear — a shadow song that plunks out its tune in the player piano of our auditory system.

“Isn’t this crazy?” he shouted over the thunder of the bass. “We’re close to being able to reconstruct the actual music heard in the brain and play it. If we had several times more electrodes, I bet we could do it.”

But for Schalk — and many others in the field — the ultimate goal is not music. It’s language. Schalk dreams of letting people speak with their neurons, issuing silent commands to their machines. You could imagine the word “cat,” say, and it would pop up on your computer screen. The areas involved with imagined speech take up just a few centimetres in the brain. With better implants, Schalk said, he might be able to pick up a word that his volunteer beams at the computer. Even with Wednesday’s implants, he and his colleagues are getting closer. One epilepsy patient moved a ball across a computer screen simply by imagining either an “ooh” sound or an “aah” sound. It marked one more step toward telepathy with machines.

For years, computers have been creeping ever nearer to our neurons. Thousands of people have become cyborgs, of a sort, for medical reasons: cochlear implants augment hearing and deep-brain stimulators treat Parkinson’s. But within the next decade, we are likely to see a new kind of implant, designed for healthy people who want to merge with machines. With several competing technologies in development, scientists squabble over which device works best; no one wants theirs to end up looking like the Betamax of brain wear. Schalk is a champion of the ECoG implant because, unlike other devices, it does not pierce brain tissue; instead it can ride on top of the brain-blood barrier, sensing the activity of populations of neurons and passing their chatter to the outside world, like a radio signal. Schalk says this is the brain implant most likely to evolve into a consumer product that could send signals to a prosthetic hand, an iPhone, a computer or a car.

“The burr hole in the skull will be small,” Schalk told me enthusiastically, as if urging me to get one of the plugs. The first dedicated trials in human beings, he says, are only a few years away. Schalk first began working with the ECoG implant in 2003, when a surgeon at Washington University in St. Louis invited him to visit the epilepsy ward; four patients had been taken off their medication and had portions of their skulls removed so they could be implanted with ECoG devices.

The implants — usually worn for about a week — allow surgeons to study the aberrant brain patterns of patients as they go into seizure and then cut out the damaged brain tissue. Schalk camped out in the Missouri hospital to wait for the periods when patients were between seizures, at which point he would try to transform the brain signals emitted by their thoughts into software commands. He was, in effect, designing a button that the mind could push. “We had no clue what we were doing,” Schalk says.

On the first day of the experiments, he sat beside a young man who gamely waited to follow instructions. As researchers rolled a monitor up to the bed, Schalk told the patient, “Now you’re going to move this cursor by thinking.” For a few minutes, the guy floundered. And then, Schalk says: “Boom, the cursor hit the target. Everyone was ecstatic.”

©2011 The New York
Times News Service