By Sara Miller, NoCamels -
By the time Sara Smolley was born, her grandmother was unable to talk to her. She had been diagnosed with early onset Parkinson’s disease at the age of just 40, which stripped her of much of her motor capabilities.
“More than anything, it was her speech that was impacted,” says Smolley, the co-founder and VP Partnerships of Voiceitt, an Israeli speech recognition platform for people who have non-standard speech.
“Of course, that affected her ability to build a relationship with my brothers and me, as we couldn’t understand what she was saying,” she tells NoCamels.
The AI-powered Voiceitt platform is designed to recognize and translate speech by people with an underlying medical condition, disability or age-related condition that means their speech is hard to understand.
Smolley’s regret over her inability to communicate with her grandmother inspired her to create Voiceitt. And co-founder Danny Weissberg, a veteran software developer, had a similar experience with a very close relative who suddenly lost her ability to communicate vocally after she suffered a stroke.
As a result of their personal experiences, they came to realize that with enough time and close contact, caregivers could indeed understand family members who could not fully use their voices.
“Just like a mother understands her child,” Smolley explains. “It became our mission to help people communicate by voice.”
The Voiceitt platform is web based, which means that it can be accessed from any internet-connected device without having to download a program or app.
It works either as voice to text or voice to synthesized speech, with the latter allowing the user to speak in person in real time, as part of a face-to-face conversation, or in a virtual, online meeting.
The technology is based on machine learning and speech recognition algorithms that are customized to the user, allowing the platform to assimilate each user’s unique way of speaking.
“We realized pretty soon that this idea of enabling in-person communication is transformative and very empowering for people with speech disabilities,” Smolley says.
“In this new world of voice-activated smart homes, smart cars, smart speakers, smart appliances, we could do a lot more by not just building a product, but essentially opening the world of these new technologies for people with speech disabilities.”
The platform asks a new user to repeat dozens of words, phrases and sentences in order to learn their speech patterns, and, says Smolley, “pretty much the person can use the software right away.”
But even after the original training process, the machine learning algorithms continue to develop and adapt to the user’s speech as they gather more data.
“Speech recognition is extremely complex and requires a lot of data and a lot of underlying sophisticated technologies,” she explains.
The company spent years developing the technology, and is already on its second incarnation, launched about six months ago after 18 months of development. According to Smolley, the new platform is more advanced and can more seamlessly translate a person’s speech.
“[Speakers] can use the technology pretty much the way a standard speaker might use dictation or speech recognition, by saying any word they want and any sentence in their language, enabling really spontaneous and fluid communication,” she says.
The new edition also shortens what Smolley calls the “onboarding” process, with users now asked to repeat up to just 200 phrases, compared to the average of 800 to 1,000 required by the initial technology.
The training session includes a hands-free version, so the mobility impaired users do not have to click to move between one phrase and the next. There is also a simpler word bank for people who struggle with longer phrases.
The platform also allows for basic spoken commands, such as “undo,” “copy” and “paste.”
Voiceitt Product Manager Michael Cash, who has cerebral palsy, tells NoCamels that the platform was specifically created with users with physical disabilities in mind. In fact, he says, he played an integral role in the development of the new platform – in particular its continuous speech feature.
“I’m able to dictate emails and work-related documents, which is a lot quicker than typing on a keyboard,” Cash says. “I can even use it on my phone to send WhatsApp messages and emails.”
Smolley says that making the user interface accessible was crucial for the team.
“It’s not just about the deep [learning] technology,” she says. “When it comes down to it, can a person with a disability use it independently?”
She points out that people with a speech disability often have other comorbidities or other motor control challenges, such as cognitive, mobility, dexterity or visual impairments.
“We’ve really invested effort and thought – working together with speech-language pathologists, occupational therapists, other stakeholders in the community, clinicians and otherwise – to make the product itself very accessible for someone with complex needs,” she says.
New users are even asked to indicate which disabilities they have so that the platform can understand who its users are and how best to accommodate them.
The company started out with grants from the Israel Innovation Authority, the branch of the government dedicated to advancing the national high-tech sector, and the BIRD (US-Israel Binational Industrial Research and Development) Foundation, whose mission is to encourage R&D that would benefit both countries.
Later, it received investment from tech heavy hitters, including Microsoft, Amazon and Cisco, as well as from the AARP (formerly the American Association of Retired Persons), a US organization focusing on issues affecting the over-50s.
Smolley says that Amazon, Microsoft and the rest of the “Big Five” tech giants – Apple, Google and Meta – are working on their own speech accessibility project. But so far, she says, this is solely focused on collecting voice data from people with speech disabilities to help improve their existing platforms rather than creating something new with a specific objective of “translating” speech.
The fact that the world’s major tech companies are working together on the project shows how difficult and complex the issue is, she says.
As such, she explains, the Voiceitt platform is unique in that it already has massive amounts of data to work with and is constantly expanding these data sets. A lack of such data, she says, is a barrier to entry.
Cash says that in the six months since the upgraded platform went live, the company focused on relationships with companies and organizations that want to acquire Voiceitt for their users, as well as institutions seeking to embed the technology into their own platforms.
“What is so meaningful for us is through communication to allow the person to be their best selves, whether it’s at work or in their personal lives,” Smolley says.
“And communication is so pivotal to that.”
