Jin Schofield (left) and Sarvnaz Ale Mohammad display the code behind ConchShell and the device itself! (Jin Schofield/Project ConchShell)
Jin Schofield and Sarvnaz Ale Mohammad have been busy during the pandemic. The two 17 year-olds from Richmond Hill, Ontario have spent the past year programming, designing, and building an incredible device.
The ConchShell, a sign language translator!
Worn as a bracelet, the gadget can be worn by Deaf people as a way to immediately turn their signing into audio speech that others can hear. Amazing! Watch them introduce the ConchShell in the video below.
So what led these two to build the ConchShell in the first place? Let's have a look.
Reading the signs
There are around 466 million people around the world who are Deaf or hard of hearing. Though this is a diverse community, many use a form of sign language to communicate. Sign language involves making hand signals that represent letters, numbers, words, and phrases. One of the most common sign languages is American Sign Language (ASL).
A sign language is a terrific way to communicate ... as long as the other person knows the language as well. Unfortunately, many people never learn sign language. This means that many Deaf people must use a translator—a person who also knows the language—to relay their thoughts and ideas to others.
This was something that Jin noticed first hand. In an interview with CBC, she said that in Grade 9 she worked on a lab assignment with a Deaf student who used a translator because Jin did not know sign language. One day, she was relaying the experience to her friend Sarvnaz and mentioned that it would great if there was a camera that could translate ASL. Sarvnaz agreed and the idea for ConchShell was born!
Time to research
Jin and Sarvnaz spent time researching ASL before programming their device. (Raspberry Pi)
It's a big leap to go from an idea to an actual working device. The two teens had some knowledge of programming, but they would need to know more. Fortunately (in a way), the pandemic gave them a lot of the time they needed to learn a new skill!
Jin and Sarvnaz both enrolled in an eight-week machine learning course. They reached out to university professors who gave them helpful advice and tips. They consulted with members of the Deaf community and with those who knew ASL to get a better idea of how the language worked, as well as what language aid devices already existed and how they could be improved.
And they got to work!
Time to build
The Raspberry Pi Zero W is the hardware that they used inside the ConchShell bracelet. (Raspberry Pi)
Gradually, the device changed from being a chest-worn camera to one that fit on a bracelet.
They used a open-source (meaning anyone can use it for free) program from Google for the software (or the computer code). The hardware (the physical computer chips and memory) was made by a company called Raspberry Pi. Now Jin and Sarvnaz were able to feed their gadget with instructions and around 30,000 examples of ASL in action!
To use the ConchShell, a person places it on their non-dominant (least used in signing) wrist. They then hold their dominant wrist in front of the ConchShell's camera and begin signing. The device uses the examples of ASL loaded into its memory to compare to the hand signals it is seeing in the camera. It can translate them into audio, which is played through a nearby computer.
Translation: success!
Always improving
Right now, the ConchShell is about 94 percent accurate, which is very impressive. After all, most similar gadgets on smartphones, such as speech-to-text translators, are far from perfect. And those are made by the biggest tech companies on the planet!
Of course, Jin and Sarvnaz are far from satisfied. According to their Youth Science Canada project page, they want to add other sign languages to the device, improve its ability to learn from experience (using AI to become better at translating the more it is used), create a smartphone app, and more!
So far they have won around $15,000 in prize money that they can put toward their project. We can't wait to see how it all turns out.
Congratulations to them both!
A wonderful effort by these two young women! Thinking a new idea using one’s imagination and we have a new invention that helps millions!
Congratulations!