In 2005, Ann Johnson experienced a stroke that left her severely paralyzed and unable to speak, at just 30 years old.
Although she could only manage sounds like “ooh” and “ah,” her brain remained active.
Now, 18 years after Johnson’s stroke, an experimental technology has achieved a scientific breakthrough by translating her brain signals into audible words, allowing her to communicate through a digital avatar.
This pioneering technology, developed by researchers from the University of California, San Francisco, and the University of California, Berkeley, relies on an implant positioned on Johnson’s brain’s surface, targeting regions associated with speech and language.
Implanted during surgery last year, the device includes 253 electrodes that intercept signals from thousands of neurons in Johnson’s brain.
Alongside the implant, doctors installed a port in Johnson’s skull, connecting to a cable that carries her brain signals to a computer system.
Using artificial intelligence algorithms, the computers interpret these signals into spoken sentences, articulated by a digitally animated figure on a nearby screen.
When Johnson attempts to say something like “Great to see you again,” the avatar vocalizes these words.
This system has proven to be notably faster and more accurate than previous technologies attempting similar feats, enabling Johnson to communicate using a relatively extensive vocabulary.
The researchers personalized the avatar’s voice using recordings of Johnson speaking at her wedding.
Additionally, the technology translates Johnson’s brain signals into facial expressions such as pursed lips and emotional displays like sadness or surprise.
Published on Wednesday in the journal Nature, the study’s results were met with great enthusiasm by Dr. Edward Chang, who performed Johnson’s surgery and serves as chair of neurological surgery at UCSF.
“I am absolutely thrilled to witness her communication through the avatar,” Chang expressed during a news briefing.
“There’s nothing that can convey how satisfying it is to see something like this actually work in real time.”
Johnson’s speech attempts were translated into words at a rate of nearly 80 words per minute, though natural speech typically ranges from 150 to 200 words per minute.
The system achieved a median accuracy of about 75% when Johnson utilized a vocabulary of 1,024 words.
Reflecting on her experience, Johnson shared emotional sentiments in a feedback survey about hearing the avatar speak in a voice resembling hers.
She recounted the challenges in the seven years following her stroke, relying solely on a letterboard and her husband to interpret her messages.
Looking ahead, Johnson expressed aspirations to become a counselor and utilize the technology to communicate with clients, suggesting that the avatar could help clients feel more comfortable.
However, the technology is not yet wireless, limiting its integration into Johnson’s daily life.
In a parallel study also published in Nature on Wednesday, researchers at Stanford University enabled a woman with Lou Gehrig’s disease to communicate in near real-time.
Using brain sensors, they developed software to decipher signals from individual brain cells into words displayed on a screen.
This study underscores the ongoing advancements in brain-computer communication technology, aiming to restore fluent conversation for individuals with conditions like paralysis.
While acknowledging these breakthroughs, an editorial accompanying the studies highlighted challenges in making such technologies widely accessible, particularly concerning operation outside of controlled research settings and for individuals with severe motor limitations.
Dr. Jaimie Henderson, the neurosurgery professor at Stanford leading the second study, remains optimistic about the future improvements and the potential to sustain communication abilities for people affected by conditions like Lou Gehrig’s disease.
“The goal is to ensure that individuals with these conditions never lose the ability to communicate,” Henderson affirmed.
