Have you ever had a nightmare, where you’re aware of everything that’s going on around you, but you can’t move any part of your body or make a sound? Anybody who has this nightmare immediately feels a sense of despair and frustration to their core. Unfortunately, there is a clinical condition with strong resemblance to this nightmare known as the locked-in syndrome (LIS). It is a neurological disorder commonly caused by a brainstem stroke or a neurodegenerative disease known as amyotrophic lateral sclerosis (ALS). It is characterized by paralysis of voluntary muscles, except for those that control vertical eye movements. People with LIS are fully conscious and have normal cognitive abilities but they can’t move or speak. It is like being imprisoned in your own body, hence the name. 

The only way of communicating for LIS patients is through eye movements and blinking. A common communication practice for these patients is to wear eye tracking devices while they move their eye gaze across an alphabet board to spell words or sentences, but it is a very slow and inefficient way of communicating as it demands good concentration and memory. The good news is that much more sophisticated assistive technologies based on brain-computer interfaces (BCI) are on the way for LIS patients, which use signals recorded directly from the brain to control a computer or a device. 

In the last few years, there has been huge progress in the development of BCI technologies. Two groundbreaking BCI studies by two different groups of researchers were published recently (1, 2). In these studies, the involved patients lost their ability to produce intelligible speech, but they could still attempt to produce speech. Each time they attempted to make a speech sound, the regions of the brain that control articulatory movements (e.g., movements of mouth and vocal cords) generate electrical signals. Neurosurgeons implanted tiny sensors, only a few millimeters in size, in the outermost layer of the brain in these speech-related regions. As the patients attempt to make a speech sound, these sensors record the brain signals, deliver them to a computer, and an artificial intelligence algorithm estimates which brain signal is associated with which sound. A sequence of estimated speech sounds is then fed to a large language model, which is trained on which sound sequence makes which word, and which words come after each other. This way, the brain signals of the patient are converted into meaningful sentences and displayed on a screen as text. 

Illustration of the neuroprosthesis for avatar animation and speech synthesis. Image from Metzger et al., 2023.

During a natural conversation, speech rate is approximately 160 words per minute. These latest BCI technologies approach 60-70 words per minute, tenfold faster compared to using eye tracking and an alphabet board. There are voice banking services for ALS patients, which saves the patient’s voice in an archive before they lose their ability to speak. When the patient can no longer speak, these archives will make it possible to speak their intended words from a device in their own voice. Moreover, new technologies investigate not only how to decode attempted speech but also facial mimicry, such as smiling, smirking or raising the eyebrows. Soon, it will even be possible to design virtual avatars in the patient’s own image and voice, which would enable an almost full body embodiment.

At the moment these BCI technologies require surgical operations to record signals from the brain. They are still in the development stage and only patients who volunteer have access to them. Surgically implanted sensors provide much more robust signals because they are placed very close to the neurons that generate these signals. Ideally, in the near future techniques that use sensors placed on the scalp without requiring surgery will be developed to yield similarly robust outcomes. Future BCI-based speech prothesis will bear less risks for patients, will be more affordable and accessible, and above all, they will decode the unspoken, revolutionizing communication for those who are locked-in.

References:

  1. Metzger, S.L., Littlejohn, K.T., Silva, A.B. et al. A high-performance neuroprosthesis for speech decoding and avatar control. Nature 620, 1037–1046 (2023). https://doi.org/10.1038/s41586-023-06443-4
  1. Willett, F.R., Kunz, E.M., Fan, C. et al. A high-performance speech neuroprosthesis. Nature 620, 1031–1036 (2023). https://doi.org/10.1038/s41586-023-06377-x