Following her deep interest in autism spectrum disorder (ASD), Usha Rengaraju has set about using AI to uncover early traits of the syndrome and to provide computer-based training tools for individuals. Specific traits that present as speech characteristics, facial expressions, patterns in text communication, as well as input from MRI scans, provide indicators that aid in ASD diagnosis. These indicators can lead to prompt treatment, which can be more effective when delivered at an early age.
The convergence of neuroscience and AI has made it possible to explore ASD with fresh eyes. A better understanding of how the human brain works can provide new insights and reveal ideas for novel research methods. Although the understanding of autism has increased in recent years, there is significant work required to get to a point that neurodivergent individuals are no longer considered out of the mainstream.
Background
Usha is the head of data science research at Exa Protocol and serves as adjunct and guest faculty member at various universities in India. Her specialty is deep learning for natural language processing (NLP), and she also has a strong interest in computer vision. "I enjoy various use cases around social good and the impact of my work gives more purpose and meaning to my life."
Accessing data for training models has proven a difficult challenge. "The biggest technical challenge lies in data privacy and security," she said, "especially in the healthcare domain. Data collection is a monumental challenge in certain use cases. I have been using open source datasets for most of my use cases because creating custom datasets for use cases around autism takes a lot of time and money. There is also a challenge in getting consent from the attendees because autism is considered taboo in certain countries."
Project Description
"My projects revolve around applying AI for autism," Usha said. "Autistic individuals have trouble understanding emotions in multiple modalities, like text, voice, and facial expressions. Deep learning for computer vision and NLP can address all these modalities. Early detection of autism is crucial for intervention programs, which in turn reduces the progressive development of symptoms."
She has been a strong advocate for ASD education for years and is also an organizer of related events. She organized the annual NeuroAI Symposium—the first event of its kind in India—linking neuroscience and data science. Usha also organized the Neurodiversity India Summit, bringing together neurologists, neurodivergent individuals, caregivers, and AI researchers. Neurodivergent individuals have unique methods of processing information and coping with different situations. Recognition of ASD traits and empathy are important steps in developing treatments. In a recent TED talk called "Speech Language Pathologist," Cynthia Coupé said, "Chances are if people aren't identified, they don’t know what their struggles are and that leads them not to seek services. While formal diagnosis may take time to catch up with the need, just being able to self identify can make a huge difference."
Given that there is no widely accepted method for diagnosing ASD before behavior traits start developing at age two, there is a strong incentive to develop an early detection method to enable early intervention. This in turn can play a significant role in the brain development and education of children with ASD.
AI can help infer emotions in text, speech, and facial expressions using deep learning algorithms. As an element of social communication, recognition of facial emotions is considered an important marker for diagnosing ASD. Using AI, the training helps autistic individuals recognize emotions in multiple modalities, such as text, speech, and facial expressions.
"I have completed the proofs of concept for all four use cases," Usha said, "and the next step would be to convert it into a mobile application that can then be used by autistic individuals."
Methodology and Approach
Early Detection of Autism Using MRI Scans
Harnessing MRI data effectively can be streamlined by AI. In this instance, the Autism Brain Imaging Data Exchange (ABIDE) dataset is used, which contains neuroimaging data from 539 individuals diagnosed with ASD. There are also 573 typical controls in the data. The deep learning architecture from the paper CASS: Cross Architectural Self-Supervision for Medical Image Analysis serves as a basis for modeling.
Facial Emotion Detection
The AffectNet dataset is used to access over a million facial images collected from the internet—querying three major search engines using 1250 emotion-related keywords in six different languages. The deep learning architecture from the paper Distract Your Attention: Multi-head Cross Attention Network for Facial Expression Recognition is used for modeling.
Text Emotion Detection
The GoEmotions dataset is used, which is a human-annotated dataset of 58,000 Reddit* comments extracted from popular English-language subreddits. There are 27 emotion categories in the dataset. The deep learning architecture MobileBERT is used for modeling.
Speech Emotion Detection
The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) dataset is used, which contains 7,356 audio files labeled with different emotions in speech (calm, happy, sad, angry, fearful, surprise, and disgust) and song (calm, happy, sad, angry, and fearful). The deep learning architecture tf-wav2vec2-base is used for modeling.
The project falls under open source research and the source code for all four use cases is in Usha's personal GitHub* repository. These four use cases are implemented using Keras, Keras Core, and Intel® Extension for TensorFlow*. Anyone interested is welcome to visit and contribute to the project.
Advice for Other Developers
The projects provided by Usha represent a good start for addressing the challenges of improving communication with neurodivergent individuals. These projects satisfy the need to establish a viable framework for further work in this area.
State-of-the-art (SOTA) deep neural networks are generally the best models you can select for a given task. "I would go for SOTA models," Usha said, "instead of implementing the papers from scratch. SOTA models in most cases can guarantee performance and save you a lot of time."
Usha also recommends Papers with Code, Kaggle*, and websites of top-tier conferences. This includes the Conference on Neural Information Processing Systems (NeurIPS), Computer Vision and Pattern Recognition Conference (CVPR), and International Conference on Computer Vision (ICCV) for the recent advancements in speech, text, medical image segmentation, and clinical datasets related to the use case.
Intel® Tools Involved in the Project
Usha used the Intel Extension for TensorFlow to implement the AI for her programs and plans to use TensorFlow Serving to deploy servers for the models that are in production. Intel Extension for TensorFlow is a heterogeneous, high-performance deep learning plug-in based on the TensorFlow pluggable device interface. This extension aims to bring Intel CPU or GPU devices into the TensorFlow open source community for AI acceleration. It allows users to flexibly plug an XPU into TensorFlow on demand, exposing the computing power inside Intel hardware.
The platform for the projects is Microsoft Windows*. The primary Intel components used include:
- AI Tools from Intel
- Intel® Developer Cloud
- XGBoost optimized for Intel® architecture
Keras 3 was chosen as the framework because of its support for multiple back ends, including JAX, PyTorch*, and TensorFlow. Because it is framework agnostic, Keras delivers flexibility and a broad scope for open source projects. Gemini* Pro and Gemma, both LLMs from Google*, were also used.
Closing Thoughts
During the Neurodiversity Summit India 2023, Usha had the opportunity to host nonspeaking autistic individuals, and in the process gained a sense of the types of challenges they face and the way they navigate them.
"Using AI to help autistic people navigate their everyday challenges is a step towards making the world inclusive," Usha said. Over the last two years, she has given talks to organizations—including Walmart* India, Infosys*, IBM*, Fiserv*, Wipro*, Capco, Wells Fargo*, Adobe*, and more—on the value of using AI for diagnosing autism.
"I have been using these projects to create awareness in several organizations on autism in India. Awareness is an integral component for autism advocacy, and I am able to combine my professional expertise with social cause and contribute to society in a meaningful way."
Commentator and writer Paul Collins framed the problem this way, "Autistics are the ultimate square pegs, and the problem with pounding a square peg into a round hole is not that hammering is hard work. It's that you're destroying the peg."
Resources
Autism Spectrum Disorder 2023: A Challenge Still Open provides an update on the relevant research in those fields in which discoveries continue to be made. Neurobiology, communications, and long-term treatment are among the areas surveyed.
Neurodiversity: The New Normal is a TED Talk that offers a view of the current understanding of ASD and the benefits of education.
The Autism Research Institute hosts education, webinars, and seminars, as well as conducting and funding research in this area.
ASD is discussed in this broadcast from the Mayo Clinic with neuropsychologist Dr. Andrea Huebner.
Usha Rengaraju’s site on GitHub includes her foundational work on ASD.
The following posts, written by Usha, offer more detail on the projects:
- MRI Classification Using Intel oneAPI and Keras
- Audio Emotion Classification Using Intel oneAPI and TFWav2Vec2Model
- Text Emotion Classification Using Intel oneAPI, Gemma, and Gemini Pro
- Face Emotion Recognition Using Intel oneAPI and Keras
Intel Developer Cloud offers free access to code from anywhere, offering cutting-edge Intel CPUs, GPUs, FPGAs, and preinstalled Intel® toolkits that include tools, frameworks, and libraries.