‘Google Libras Translator’ connects people with hearing impairments to the internet

Before completing his degree in social communication, businessman Ronaldo Tenório studied computer science for three years. In professional life, he didn’t have to choose: he joined the two passions in an application that is a kind of “Google translator” for people with hearing disabilities: Hand Talk.

Founded in 2013, the startup works on two fronts. One of them is an application that translates texts or speeches into Libras (Brazilian Sign Language), through animation. The other is a corporate solution: a plugin for websites that want to make their content more accessible.

“We realized that the internet was practically offline for the deaf”, says Tenório. “There are studies that show that 80% of deaf people have difficulty understanding the written language of their country, as their first language is sign language.”

The idea, he says, arose in a college class in 2008, when he needed to develop a project to solve a communication problem. In that beginning, when the smartphone was just created, I thought of something just for the computer. “This idea was kept for four years. I finished college and went into business, founded a communication agency”, he says.

The idea came out of the drawer when, years later, Carlos Wanderlan, a colleague who had just taken a course in application development, sought him out to put his newly acquired knowledge into practice. They completed the team with Thadeu Luz, a special effects specialist, and founded the company.

In 2013, they won the Social Entrepreneur of the Future Award, held by sheet, and joined the Network sheet of Social and Environmental Entrepreneurs.

The app is free, although there is a paid version for those who want to do without advertising. In addition to being an educational tool, as it allows users to learn signs, the tool can enable quick communication with a deaf person.

The plugin is offered for companies that want to give pounds option on their websites. With the device activated, whoever is on the page can click a button, which opens a window where the virtual translators Hugo and Maya translate the text into Libras.

To develop the technology, the first step is to record videos of interpreters making the signs. A software captures the models’ movements and facial expressions, which improve the signs that already exist in the company’s database or add new ones.

“We have thousands of signals at the base. Each unit of this signal went through an interpreter and a 3D animator making the corrections. Everything goes into that cake of information, it’s our translator’s raw material. This goes through artificial intelligence to get out the logical sequence of translation”, he explains.

This is work that has no end: as long as people use the language, it will be modified and updated. When entering niches with very specific vocabulary, such as the legal sector, for example —in the case of a website that they serve—, it is necessary to revisit this base to check if the signs are adequate.

The improvement makes the movement of virtual translators more human. A comparison is the evolution of the voice of virtual assistants, which has gained more intonations over the years.

One of the entrepreneurs’ challenges was precisely to capture the interpreters’ facial expression, which carries meaning at Libras. The “house” sign, for example, is made by putting the fingertips together, stretched out, like a roof. A large house could be expressed through this same sign, but by raising the eyebrows.

“As a human being, we have all the possibilities, but a robot has its limitations”, says Tenório. Today, however, artificial intelligence already captures all points of the face to identify movement.

Regionalisms also exist, like all languages. “Father”, in most parts of Brazil, is expressed by two signs: running the fingers along the side of the chin, like a beard, and then kissing the back of the hand, like the blessing that was asked of parents. In parts of southern Brazil, however, the first sign is made by the index finger in the upper lip, simulating a mustache.

The option is to try “to find a more unified way of communicating, without forgetting that there are variations”, says Tenório.

Currently, the company has reached a point of financial balance and is looking to expand. The app already offers ASL (American Sign Language) translations and recently launched Community Hand Talk, a collaborative data-gathering platform where sign language speakers from around the world can upload videos and confirm translations. .

They intend to use this data to improve the languages ​​they already operate in and to release other languages. “There are more than 200 sign languages ​​in the world. People who want to take Hand Talk to their countries are able to contribute”, he says.

Along the way, Tenório learned Libras to communicate with employees and friends. “I learned that the world was much bigger than the bubble I was living in,” he says. “I realized that deaf people live like foreigners in their own country, where hardly anyone can communicate with them, where their parents, their co-workers, have no dialogue with them.”

Eventually he even forgets a sign. “It’s a challenge. But when I don’t know, I consult the application,” he says.