Tag: tobii dynavox

If You Want To Sing Out…

On February 26, 2026

I invited my husband, Barry, to be a guest blogger for this piece, because I can’t begin to explain how he achieved this miracle. Please note: Barry is a Principal Software Engineer by day. I will attempt to translate anything I think may trip up us lay people.

Jessie did some “phrase banking” when she was first diagnosed with ALS in 2017 in anticipation of losing her voice. She recorded a library of phrases that are stored for verbatim use on a text to speech device. When she types “Hello!” the device plays her banked recording of “Hello!” in her own voice.

Jessie intended to do “voice banking” as well, in which she records a huge quantity of sentences, and then a company generates a synthesized voice that sounds like Jessie’s original voice. Unfortunately, between teaching and looking for a new house, Jessie only got halfway through recording the sentences before her speech was affected.

Like most folks in this situation, she listened to the various canned voices on her Tobii Dynavox device and chose a female voice that sounded “close enough”. These voices are good. The technology has come a long way from the computerized voices we heard growing up (…think “War Games” or Stephen Hawking’s early use of text to speech technology) but it was never her voice.

Well, we recently heard an interview with Eric Schmidt, former CEO at Google. He was talking about how they had a prototype a while back that could create a realistic voice clone from no more than a few minutes of audio. It was not released at the time for various reasons but it got me thinking that the time was right for us to return to this issue and do something!

As you know, Jess was a public school teacher for 24 years and she had a number of classroom videos she filmed. I extracted a bunch of clear voice audio from these and fed them into the ElevenLabs voice cloning app (https://elevenlabs.io/).

ElevenLabs markets themselves primarily for agentic (autonomic AI) content creation: voice overs, audiobooks, podcasts, chat bots, etc. However, their cloning capabilities were reviewed as highly accurate and expressive.

Indeed, the voice modelling was spot on from the ~60 minutes of input we provided. We now had a nearly perfect sounding Jess-bot!

I thought at this point that we might use the voice clone to generate content that could be fed into the traditional voice banking apps, having the voice clone do the “recordings” that you would normally do yourself in this process. Unfortunately, there is a maze of intentional licensing barriers to such an approach.

However, ElevenLabs does have a partnership with BridgingVoice which is a non-profit organization focused on ALS and assistive communication technology. “We help people living with ALS (pALS) maintain their ability to meaningfully communicate with family, caregivers, and medical professionals, no matter the stage of their disease.” BridgingVoice picked up the licensing fees for our use of the ElevenLabs voice clone API (Application Programming Interface). (https://bridgingvoice.org)

Then I wrote a little application to act as a broker between the various components. It takes in the input text from her typing app, communicates with the ElevenLabs API to vocalize that text, and processes and plays the audio returned. (From Jessie: if this paragraph confuses you, just go on to the next one. It won’t affect your understanding.)

And voila, Jess has her voice back! She types (with her eyes) anything she wants to say and it is vocalized in near real time with her cloned voice.

Note that BridgingVoice has a Tobii Dynavox integration app similar to the one I wrote that they provide free of charge to the pALS they support. If you need this, you should totally use it – it works just fine. I decided to write my own software because…

I wanted to learn more about the technology and I was already halfway down the road of thinking through various issues in my head, so I couldn’t stop myself. 🙂
The BridgingVoice app had some volume integration issues I really wanted to address to make the user experience smoother.
I’m sure the ElevenLabs Text to Speech API is going to evolve quickly and I wanted to be in a position to incorporate new features without annoying the BridgingVoice folks with lots of questions and feature requests.
I also added a bunch of additional INFO and DEBUG diagnostic logging so I can help Jess when things aren’t working so well. (From Jessie: I ignore the technical terms I don’t understand, if it doesn’t get in the way of my overall understanding.)
2026-02-22 13:41:04.202 [INF] VoiceApp Main()…
2026-02-22 13:41:04.234 [INF] Text: 86 characters
2026-02-22 13:41:04.867 [INF] GetAudio(): 0.627 seconds
2026-02-22 13:41:10.624 [INF] PlayAudio(): 5.754 seconds
2026-02-22 13:41:10.628 [INF] VoiceApp Main(): 6.516 seconds

The “Speak” button to the right on her eye gaze keyboard vocalizes with the same canned voice as before. The new “Voice” button on the left vocalizes with the clone. But the clone is dependent on a network connection to access the ElevenLabs API so we left the Speak button in place as a backup when we’re out of the house, or having a Wi-Fi outage, or whatever.

(Also, the clone tends to add confusing emotional tones to texts with just a word or two in length. So the old voice or banked phrases are still much better at saying “Yeah” or other simple texts that don’t provide enough language context for the clone to model very well.)

This is a small clip of Jessie from 2011 leading a class of high schoolers through a reading of Homer’s Odyssey:

And this is Jessie 2.0 today:

(Ok. You got me. I typed that, not Jess. But you get the idea.)

I’m sure this process is going to get even easier from here on out. It’s not hard to imagine a time when you turn on your text to speech device for the first time and it prompts you to record or input a few simple voice prompts. And that’s it, you have a highly accurate personalized digital voice. I’m just glad we were able to manually connect the tech dots for Jess’s benefit today.

So let Jessie know if you’d like to come by for a visit some time soon. She’d be happy to catch up and talk with you in her own unique and beautiful voice.

…Sing Out.

– barry

Welcome to My Eye Gaze World

By Jessie

On May 29, 2025

In living with ALS

I took it for granted that you all know that I use eye gaze technology (I type with my eyes) to communicate in all forms: speaking, texting, and writing everything from emails to this blog post. That is, I took it for granted until my sister, Sephrah, said something that made me realize that only the people who have seen me within the past four years know my reality. In honor of ALS Awareness Month, I will now attempt to show you what it’s like to rely on eye gaze technology.

I have to type with my eyes, because I’m completely paralyzed except for a little bit of movement in my legs. I can still feel everything, which makes itches a lot of fun.

Before I continue, I want to clarify that I didn’t get ALS and immediately go to eye gaze. First, I was still able to type with my hands. As my right fingers became weaker, I did a modified version of touch typing with just my left hand. That lasted for about a year. When my hands got too weak to type, I used a voice to text software, which worked well, but I couldn’t use it around other people, both because the software wouldn’t work well with ambient noise, and because I was still teaching, and had to write confidential information about students.

My next phase started when ALS affected my speech so much that the software wouldn’t work. Jackie, the tech specialist from the ALS Association, came to my house and attached a camera to my laptop and put a silver dot in the corner of my glasses. This system let me manipulate the cursor by moving my head around to select the letter or icon I wanted, and then click by waiting for a few seconds. It was my favorite accommodation. It was fast, and the process led to few mistakes.

Eventually my neck got too weak for this system. Enter a tobii dynavox touch tablet with built-in eye gaze technology and a giant clicker button, which I held on my lap. I would use my eyes to select what I wanted, and press the button on my lap the way most people double click a mouse. Eventually, I couldn’t do the button anymore. I’m 100% eye gaze now.

Every time someone rolls me up to my device, the first thing we have to do is track my eyes. That means we line up the machine with my eyes. Here’s what the tracking screen looks like.

The white circles are my eyes and they have to be centered, and the white arrow has to be level with the white eyes. Distance, height, angle…it is harder than it looks, because if it’s not just so, one or both eyes will disappear, and the person who is helping me has to move the device around until it gets fixed. The lights on the bottom are what track my eyes. If they are blocked, I can’t do anything.

Next, I calibrate my eyes, which synchronizes my eyes with the machine – ideally. I calibrate by following circles around to nine points on the screen. See below for what the end result should look like. If the tracking is not perfect or my eyes are tired or watering, my eyes won’t be synchronized, and I will have trouble typing or doing anything with the machine. This usually means starting over again from the top with tracking and calibration.

This is the keyboard (see below) I use to type what I want to say and I often type other things and cut and paste into other documents, because it’s faster than the windows keyboard. I wanted to type “…I make mistakes all the time.”, but this is what I actually typed. That’s why I don’t like people reading what I’m typing over my shoulder. I make a lot of mistakes.

Notice that the question mark and other punctuation marks or numerals are not on this screen. For them, I have to switch to other screens. However, my biggest complaint about this keyboard is that there is no caps lock anywhere. The only way to capitalize every letter in a word or acronym is to do it one letter at a time and hope that I don’t select the wrong letter on my way. Imagine selecting whatever you look at. That’s my world.

It’s maddening when there is a video or animation on part of a website I’m looking at, because it is impossible for me to look away. You know the advertisements on YouTube that you can skip after four seconds? It takes me much longer than four seconds to skip the ads, because I can’t stop watching them. It’s like telling myself not to look at a car accident on the freeway; I can’t help but look anyway.

One more major complaint about eye gaze technology. No one has figured out how to get it to work with sunlight, so I can’t use it outside or in the car. That is the reason I don’t like to spend a lot of time outside, which is unfortunate, because I love nature. I need someone to create a collapsible awning, like the ones on baby strollers, to extend over my head and the device, shielding us from the sun.

I used to think that eye gaze technology had existed since the 1960s, because Stephen Hawking used it, but no. He used a muscle in his cheek to do all of his communication, including writing books! I don’t have any muscles in my face that I can control to communicate. I recently found out from Jason Becker, who has been living with ALS for 30 years, that 20 years ago, eye gaze technology was terrible. For all the complaining I do about my device, I am lucky to have access to the assistive technology available today. If I had been diagnosed even just 5 or 10 years earlier, I would be silent by comparison and certainly wouldn’t be able to write.