Executive summary
When she found that students who are blind can’t play some online games, Crystal Yang wondered whether they might be missing out on some fun educational opportunities, too.
Working with a team of volunteers, she created 16 AI-based games to help students who are blind and visually impaired with English and math.
The games use the OpenVINO™ toolkit to convert between text and speech, prompting players with audio questions and interpreting their spoken replies. The toolkit’s pretrained models helped ensure the games accommodate a range of accents.
Yang won the Intel® AI Impact Festival Award in the 13 to 18 age group in 2024, and part of her prize was an Asus AI PC, powered by Intel® Core™ Ultra processor.
Running AI models on AI PCs cut the development time for the games and enhances players’ privacy because their recordings can be analyzed locally without being sent to the cloud.
Challenge: Creating Educational Games for Students who are Blind
“I have a lunch table at school, and we love playing a popular online word game,” says Crystal Yang, founder of Audemy and high-school student. “We’re super competitive. One day, a student who is blind joined our table but couldn’t play because the game is a visual medium. So, I built her an audio version over the summer. I realized that if entertainment wasn’t accessible, what about education?”
Yang researched educational games for students who are blind and couldn’t find any. While sighted children can choose from a huge range of computer games to develop language and math skills in a fun way, Yang found that students who are blind or visually impaired were missing out. The games often didn’t work with the screen readers that help blind people and those with impaired vision by reading screen contents aloud.
Yang wanted to develop games that would help children who are blind and visually impaired to learn using an audio interface. To do that, she needed powerful voice recognition and text-to-speech capabilities.
Solution: Using AI to Convert Between Text and Speech
Working with a team of volunteers that she recruited, Yang developed 16 AI-based games for students who are blind, all inspired by student ideas. “It’s great to see their wildly creative ideas and to be able to implement them in the games,” says Yang.
“One of the most popular is the Color Game,” she says. “It helps students learn about what color different objects are. For students who are blind and visually impaired and don’t have access to colors like we do, it’s super helpful for them to know what color objects typically are.”
Other games include Definition Detective, where students are challenged to name objects described by a visiting alien; Odd One Out, where they must identify which item doesn’t belong with the others; and Fruit Frenzy, where they practice addition by counting fruits. In each case, the game speaks to the student, and the student records their reply using the microphone, making for an entertaining conversation-style game format. The game uses AI to convert text to speech and to convert the player’s response from speech to text for analysis by the game code. The games work on desktop, mobile, and screen reader devices, so it’s convenient for people to play using their favorite device.
Yang worked closely with teachers and students to ensure the games meet their needs. “Even in the ideation stage, and throughout development, we’ve been able to consult many teachers through interviews and have many user tests with students, to make sure our games are as accessible and useful as possible,” she says. “Teachers introduced me to assistive technologies, such as screen readers, which are the most common technologies that students who are blind and visually impaired use on their computers. A teacher helped me get in touch with tons of organizations for people who are blind and after that, Audemy picked up steam super quick. I realized that the community of students who are blind and teachers was really tight knit, which helped a lot.”
Yang built the software using the OpenVINO toolkit, which provides trained models for text-to-speech and speech-to-text conversion, and includes tools to optimize the performance of AI models on Intel® processors. “The OpenVINO toolkit has been beneficial in ensuring that the games are effective because the AI models run extremely fast,” says Yang. “It has been crucial in ensuring accessibility for people with different accents, and it has empowered us to connect people all around the world.”
Yang won the Intel AI Impact Festival Award in the 13 to 18 age group in 2024 and part of her prize was an Asus AI PC powered by Intel Core Ultra processors. “The AI PC powered by Intel has been a pivotal tool in my workflow,” says Yang. “It allows me to run AI models locally, which has been useful in developing Audemy.”
“The AI PC has helped increase performance for the AI tools that are slowly being integrated within technology products such as AI video editing tools,” she adds. “I can do things much faster, increasing my productivity. I would definitely recommend the AI PC powered by Intel Core Ultra processors to other developers.”
The games are available to play online, with processing of the player’s voice recording being done in the cloud. However, the games also run on AI PCs, with AI processing carried out on the device. “The data that’s collected within the games is student data, so it’s been super helpful to be able to run these AI models locally because that increases the privacy of the data for the students,” says Yang.
Technical Components of Solution
- Intel Core Ultra processor. Intel Core Ultra processors are high-efficiency processors built to deliver next-gen AI experiences in sleek and slim mobile form factors. These processors are the foundation of the AI PC, with a high-throughput GPU, low-power NPU, and fast-response CPU.
- OpenVINO toolkit. OpenVINO toolkit is an open-source toolkit that accelerates AI inference with lower latency and higher throughput while maintaining accuracy, reducing model footprint, and optimizing hardware use. It streamlines AI development and integration of deep learning in domains like computer vision, large language models (LLM), and generative AI.
Results
The games have been well received in the education community, not least because they are freely available, removing any financial barriers to adoption. They’re used by more than 5,000 students who are blind and visually impaired across 19 partnering schools. Players have answered a total of 100,000 game questions. “A lot of people emailed me to say they love the platform and it’s been great to see it grow into what it is today,” says Yang.
“One story that stands out is the first time that I got a video of a student testing out the platform,” says Yang. “He was playing an existing story-building game that we made accessible. It was so exciting to see the student laugh when he put in ridiculous words, and then at the end, he was excited to listen to his story.”
“It’s been amazing to see my project getting recognized with awards,” says Yang. “I’m so grateful because I’ve connected with Intel, which has supported me on the project.”
Audemy is used by students at home and at school, but Yang wants it to be more widely adopted in classrooms and integrated into curricula for students who are blind and visually impaired.
She is now starting a dual degree program in computer science and business. “Having business skills will help me build Audemy on the business side and scale it, while computer science skills will help on a technical level,” she says. “My dream is to be an entrepreneur. I’ll experiment with different companies and try new things, having fun and seeing where it takes me.”
Lessons Learned
The success of Audemy shows that having no budget need not be a barrier to changing lives. “It’s easier than ever to start building whatever you want to help the community with technology,” says Yang. “Intel has a lot of great tools to help students get started with learning technology.”
Yang started Audemy as a student and didn’t have the funds to hire developers or contact people to raise awareness. However, she recruited over 30 volunteers to help her realize her vision. “I have a super exciting volunteer story,” says Yang. “I didn’t have funds to hire an entire team of developers or outreach people, so I put a request on a website. Right the next day, I got 30 volunteer applications to help me build my website. What was really cool was that these people were from all over the world and from education backgrounds,” says Yang. “It was so exciting to see everyone come together and generously donate their time to help my organization.”
Audemy uses cheap web hosting, and the team stays in touch using Slack, a cost-effective communications tool. “Our costs are actually really low,” says Yang, “which is great because I don’t have access to that much funding as a student. It’s been super useful to run on low-cost infrastructure to keep Audemy going without strong financial support.”
“Using low-cost infrastructure inspires other entrepreneurs. They can hop into the venture space without needing big venture capital backing,” she says.
Find the solution that is right for your organization. Contact your Intel representative or visit intel.com/education.