Google Glass helps children with autism read emotions

Issue: September 2018

Exploratory study results demonstrated that a machine-learning-assisted software system that runs on Google Glass, paired with a smartphone app, helped children with autism understand the emotions conveyed in other’s facial expressions.

To speed up access to therapy for children with autism, researchers at Stanford developed Superpower Glass, a tool that delivers therapy at home using a machine-learning-assisted software system that runs on Google Glass paired with an Android smartphone app. The tool, designed for use during social interactions, provides audiovisual feedback to the wearer that corresponds to which of the eight emotions — happiness, sadness, anger, disgust, surprise, fear, neutral and contempt — it recognizes during social interactions via its outward-facing camera.

“Early intervention can benefit children tremendously; however, children with autism are getting diagnosed too late, then land on long waitlists for access to therapy. Therefore, many are losing opportunities to improve,” Dennis P. Wall, PhD, from the departments of psychiatry and behavioral sciences, pediatrics and biomedical data science at Stanford University, told Infectious Diseases in Children. “In addition, therapy in the natural environment tends to be more impactful than therapy provided in the clinical environment but scaling to meet demand is challenging. Mobile home-based tools can help. Thus, we designed our tool to address some of these important challenges and to begin establishing the data necessary to show that home-based wearable therapy can safeguard against children missing key opportunities.”

In this pilot exploratory study, Wall and colleagues examined whether Superpower Glass could benefit social-affective learning for children with autism. Fourteen families who had a child aged 3 to 17 years with autism used the therapy for at least three 20-minute sessions per week for 10 weeks on average. Researchers assessed changes from start to study end through parent-completed questionnaires, the Social Responsiveness Scale and a facial affect recognition task (Emotion Guessing Game).

According to a press release, the investigators designed three ways to use the face-recognizing program: “free play,” in which children wear Google Glass while interacting/playing with family and the software provides the wearer with a visual or auditory cue each time it recognizes an emotion on someone’s face; “guess my emotion,” in which a parent acts out a facial expression corresponding to one of the eight emotions and the child tries to identify it; and “capture the smile,” in which children give another person hints about the emotion they want to prompt until the other person acts it out.

After their child received the treatment, parents reported increased eye contact and greater social acuity. Findings from repeated-measures one-way ANOVA analysis showed a drop in Social Responsiveness Scale total scores by an average of 7.14 points (P < .001) and a rise in Emotion Guessing Game scores by an average 9.55 correct responses (P < .01).

“The take-home for clinicians is that our system ... is feasible and shows promise as an intervention to improve social awareness in children with autism,” Wall said. “An additional take-home is that children take well to the glasses and the augmented reality form factor, so much so that it was they who came up with the name ‘Superpower Glasses.’”

Children with more severe autism were more likely to choose the game modes rather than free play, according to the press release. In addition, none of the childrens’ social responsiveness scores increased during the study, indicating autism symptoms did not worsen. Furthermore, six of the 14 participants had large enough reductions in their scores to move down one step in the severity of their autism classification — four dropped from severe to moderate, one dropped from moderate to mild and one dropped from mild to normal, the release said.

“The children in our study improved on a standard outcome measure and showed increase in emotion recognition, increased eye contact and overall an increase/improvement in social behavior. These are positives, but we must treat [these results] cautiously,” Wall said. “The next step — which we have completed — must be to run a randomized controlled trial to validate these promising feasibility results.” – by Savannah Demko

Reference:
Daniels J, et al. npj Digital Med. 2018;doi:10.1038/s41746-018-0035-3.

Disclosures: The authors report no relevant financial disclosures.