This article is from the In-Depth Report Science and Soccer's World Cup

World Cup to Debut Mind-Controlled Robotic Suit

Scientific American speaks with the scientist behind an innovation that will enable a paralyzed person to walk and “feel” it

Jun 9, 2014 |By Dina Fine Maron

More In This Article

Effect of Panel Shape of Soccer Ball on its Flight Characteristics

“The beautiful game” will have a robotic addition at this year’s World Cup kickoff.

Clad in a robotic body suit and a cap adorned with electrodes that will detect brain signals and cue leg movement, a paralyzed Brazilian will take to the pitch and move with the assistance of a specially designed exoskeleton during the opening ceremony of the World Cup on June 12.

Legions of fans tuning in to watch the event will see the debut of a technology that, according to its creators, will one day use people’s brain waves to control robotic limbs and effectively make wheelchairs obsolete. This initial demonstration is merely an early prototype, but Duke University neuroengineer Miguel Nicolelis, the man behind the project, envisions a future in which the brain–machine interface will allow individuals who have lost mobility from accidents or disease to get back on their feet—even if a robotic suit is needed to make that happen.

The technology hinges on sensors that listen to a barrage of electrical signals in the brain, reading and translating them into digital commands that, in turn, spark an artificial device to act on the brain’s prompts.

Nicolelis wrote about his plans for this World Cup demonstration in the September 2012 Scientific American (pdf). To learn more about the current state of the technology and exactly what to expect on and off the field, SA spoke with Nicolelis just a week before kickoff.

[An edited transcript of the conversation follows.]

Several years ago you were already hoping your technology would be ready for a debut at the World Cup opening ceremonies. Can you bring us up to speed on what the exoskeleton will be able to do on June 12?
The eight patients we have worked with in the last few months, who are [mostly] in their late 20s, are able to walk in the lab with the exoskeleton and kick the ball. But they also got a sensation they were walking, which is one of the key objectives here with this project—to give them a feeling that this is not a machine carrying them but that they are actually walking. This is already happening, because not only are they controlling the movements with brain activity but they are getting feedback from the device delivered to their arms where they still have sensation. They all have this phantom sensation—like a phantom limb sensation—and that is very new. We didn’t know that would happen but that is a very important find.

For the World Cup demo we have only a very limited time so it’s more of a symbolic gesture that science can provide the kind of hope that millions of patients around the world would like to have to one day walk again. It’s actually a kickoff for our project. In 16 months we went from zero to full project—we were able to deliver this on time and show that the brain–machine interface has a really big future.

So, what will the selected individual actually be doing that day? Walking out onto the field and kicking the ball as you wrote about in 2012?
I cannot tell you what we are going to be doing. I would not tell my mother, let alone Scientific American—sorry. It’s a surprise for a billion people.

It’s certainly technologically complex to fine-tune communication between the brain and robotic circuitry. How much of the action of the exoskeleton is being controlled by the person and how much is actually automated?
Without the person there is no movement. We are using a noninvasive technique that is an EEG- [electroencephalography-] based interface. So the person has to imagine what kind of movement he or she wants to make and that decision triggers what the exoskeleton does—when it stops, when it kicks the ball. In 2002 I published a paper where I discussed this concept of what we call “share control” in which the higher order decisions are taken by the brain and lower level mechanics are taken by the robot. In real time the person is processing the feedback from the exoskeleton so there is symbiosis. I cannot tell you what percentage is from each.

Right now, is the brain sending that initial “go” signal but the actual steps and kick are preprogrammed into the suit’s computer?
We have a couple patients who modulate the EEG from step to step. We discovered some patients are actually capable of doing that and can control the velocity of the device. This is just coming out in the last few weeks. We have many things that we are going to publish throughout the next few months showing that we have actually extended the abilities of this noninvasive interface. We are basically creating a mental language for the patients to have a variety of actions that they can control and we are just in the beginning of this. The potential is pretty big.

Earlier on you thought you would have to implant electrodes directly into the brain to manipulate the robotic limb, but you did not actually end up doing that. These young adults are wearing a cap with external sensors, right?
The implantable technology is not ready. It needs to show benefits that outweigh the potential dangers of a neurosurgical procedure. I’m a very strong proponent of this technology but it’s not ready for prime time yet. It has to be miniaturized and improved. Our eight patients are very happy with the results and I don’t think they would be happy with implantable technologies that cannot deliver more benefits for locomotion than we have for external sensors.
With new implants we designed—discussed in the journal Nature Methods this month—we can now simultaneously record about 512 channels wirelessly and simultaneously, which no one has done before. These animals [monkeys] were able to control wheelchairs—they drove the wheelchairs around and did all sorts of tasks just by brain control showing that this is possible and has a very nice future, but it’s just not ready for patients right now. [Scientific American is part of Nature Publishing Group.]

The signal comes from EEG but my understanding is EEG is prone to noise and contamination, especially from the muscles and from eye movements. Have you done anything to prove that the signals are coming from the brain and not these other sources?
We are recording EEG simultaneously so we can see if there is any effect.

In theory, could the exoskeleton be controlled equally well by sound or finger movement or something like that?
I don’t know. We haven’t tried.

How did you find the Brazilian patients you’ve been working with for the last few months?
We did it with health authorities in Brazil. We partnered with the largest spinal cord hospital in the country—they have 65,000 patients—and selected the eight we decided were best for this study.

Did you want young adults because they were relatively lightweight?
They are from 22 to 38 or so, so they are in a different range of weights, and some of them are para-athletes so they are healthy young individuals.

Which technologies are novel here—the EEG cap, the suit, the software that processes signal?
I haven’t seen any exoskeleton that is controlled by voluntary brain activity and provides feedback to the patent simultaneously. That doesn’t exist—or at least it hasn’t been published in the literature.

How far is this technology from being usable by paralyzed people around the world?
It’s the same question people asked when people went to the moon, and we already know the answer. This is how science progresses. You make big advances and then you discover what can be done and then you try to apply this by developing new versions that can be used by everybody that needs it.

If things don’t go as planned on the big day and the suit does not operate according to specifications or the patient gets nervous, is there some sort of backup switch that can operate the system?
We have hours and hours of video footage of these patients walking using this device that we will make public and available, and we are hoping that the suit will work.

Is there any particular obstacle you are concerned about for that day like cell phones in the audience or anything like that?
My obstacle right now is just journalists. There are too many pessimists who cannot see the big picture of what this means for science in a developing country and who only look at scientists as a bunch of people that create bombs and things to kill people. We need to portray science as a good endeavor that can improve mankind. That’s what we are trying to do.

Why did you choose the World Cup to debut this technology?
Because the World Cup is Brazil. Brazil is the World Cup. There’s no other country that embodies the beauty of football like Brazil. It makes total sense. We have plans for the Paralympic Games, too. That’s in Brazil also. Don’t worry, we’ll be there.

YES! Send me a free issue of Scientific American with no obligation to continue the subscription. If I like it, I will be billed for the one-year subscription.
cover image