Google Glass adaptation opens the universe to deaf students

“Signglasses” system may help deaf literacy

Ordinarily, deaf students are left in the dark when they visit a planetarium.

With the lights off, they can’t see the ASL interpreter who narrates their tour of outer space. With the lights on, they can’t see the constellations of stars projected overhead. 

That’s why a group at Brigham Young University launched the “Signglasses” project. Professor Mike Jones and his students have developed a system to project the sign language narration onto several types of glasses – including Google Glass.

The project is personal for Tyler Foulger and a few other student researchers because they were born deaf.

“My favorite part of the project is conducting experiments with deaf children in the planetarium,” Tyler wrote. “They get to try on the glasses and watch a movie with an interpreter on the screen of the glasses. They're always thrilled and intrigued with what they've experienced. It makes me feel like what we are doing is worthwhile.”

Serendipitously, the only two deaf students to ever take Professor Jones’ computer science class – Kei Ikeda and David Hampton – signed up just as the National Science Foundation funded Jones’ signglasses research. Soon after the Sorenson Impact Foundation provided funding to expand the scope of the project.

“Having a group of students who are fluent in sign language here at the university has been huge,” Jones said. “We got connected into that community of fluent sign language students and that opened a lot of doors for us.”

The BYU team tests the system during field trip visits by high school students at Jean Messieu School for the Deaf. One finding from the tests is that the signer should be displayed in the center of one lens. That surprised the researchers, who assumed there would be a preference to have video displayed at the top, like the way Google Glass normally does it. Deaf participants preferred to look straight through the signer when they returned their focus to the planetarium show.

The potential for this technology goes beyond planetarium shows. The team is also working with researchers at Georgia Tech to explore signglasses as a literacy tool.

“One idea is when you’re reading a book and come across a word that you don’t understand, you point at it, push a button to take a picture, some software figures out what word you’re pointing at and then sends the word to a dictionary and the dictionary sends a video definition back,” Jones said.

Jones will publish the full results of their research in June at Interaction Design and Children. But his favorite part of the project happens after the test shows end and the high school students just get to talk with his BYU students.

“They see deaf university students succeeding and doing cool stuff,” Jones said. “It’s really rewarding."

The “cool stuff” the BYU students do comes from a variety of fields. Tyler is certified to use the university’s MRI lab and plans to attend medical school. Kei is from Japan, knows four languages and belongs to BYU’s nationally-acclaimed animation program.

And though Amber Hatch can hear, this project has furthered her ambitions to become a psychiatrist serving deaf clients.

“This project has also allowed me to utilize my ASL knowledge and to communicate with the deaf community in a way I never really thought possible for me,” Amber said. “It’s an amazing project and I am excited to see where it will go in the next year.”

For a closer look at the signglasses project, check out this video from student Austin Balaich: 

Co-authors from BYU on the study include Jeannette Lawler, the planetarium director, Eric Hintz, a physics and astronomy professor, and Nathan Bench, a post-doctoral fellow in computer science. Other co-authors include Fred Mangrubang of Gallaudet University and Mallory Trullender of Mantua Elementery School in Fairfax, Virginia.

 

More Information on This Article

Article Source/Further Information

News and Events

Image for Mystery of Haumea's Formation Solved
BYU Physics and Astronomy student Benjamin Proudfoot recently published research in the prestigious journal Nature Communications that solves the mystery of the icy dwarf planet Haumea's formation.
Image for Debunking acoustics myths around the Saturn V
When the Saturn V rocket propelled man to the moon in July 1969, the blast from the rocket’s engines was tremendous. Marked by a dazzling display of flames and deafening noise, the monumental event gave rise to widespread claims that the acoustic force of the rocket melted concrete and ignited grass fires miles away. New research from BYU debunks this common myth.
Image for Dr. Aleksandr Mosenkov, new Astronomy faculty
Dr. Aleksandr Mosenkov, new faculty, looks forward to receiving some of the first data from the James Webb Space Telescope to study galaxy formation
Image for BYU Acoustics Records Artemis Launch
A group of BYU students and professors gathered acoustical recordings of at the world’s most powerful rocket launch.
Image for Kent Gee Recognized by AIAA
Kent Gee is selected as Associate Fellow of American Institute of Aeronautics and Astronautics in their class of 2023
Image for West Mountain Observatory contributes to understand distant galaxy
BYU’s West Mountain Observatory was one of 37 ground-based telescopes throughout the world monitoring the active galaxy that is roughly 1 billion light years away.
Image for Dr. Tim Leishman retires from BYU
Dr. Leishman's time at BYU was filled with great teaching and profound mentoring
Image for Dr. John Colton: Table Tennis Champion
Dr. John Colton won the 2022 BYU intramural table tennis tournament