This article is more than 5 years old. Information may no longer be current.
Home videos may be useful in diagnosing autism
Home videos 2 minutes long were used to diagnose autism spectrum disorder in children, according to findings recently published in PLoS Medicine.
According to researchers, standard approaches to diagnose ASD assess up to 100 behaviors and take several hours to complete, which leads to a clinical backlog that can be detrimental for those ultimately diagnosed with autism.
“Behavioral interventions for ASD are most impactful when administered by or before 5 years of age; however, the diagnostic bottleneck that families face severely limits the impact of therapeutic interventions. Scalable measures are necessary to alleviate these bottlenecks, reduce waiting times for access to therapy, and reach underserved populations in need,” Qandeel Tariq, a data scientist within the department of pediatrics at Stanford University, and colleagues wrote.
Tariq and colleagues applied eight machine learning models to 116 home videos of children with autism (mean age, 4 years 10 months) and 46 videos of typically developing children (mean age, 2 years 11 months). The videos were 2 minutes long. Nonexpert raters needed 4 minutes to measure 30 behavioral features.
Researchers found that the five-feature logistic regression classifier — which used medical records generated through the administration of ADOS Module 2 — yielded the highest accuracy (area under the curve = 92%; 95% CI, 88-97) across all ages tested. A prospectively collected independent validation set of 66 videos, half featuring children with ASD, achieved lower but comparable accuracy (AUC = 89%; 95% CI, 81-95). Logistic regression to the 162-video-feature matrix to construct an eight-feature model achieved 0.93 AUC on a held-out test set and 0.86 AUC on the validation set of 66 videos. Other models — alternating decision trees, logistic regression, linear support vector machine, logistic regression, radial kernel, and support vector machine also performed well, but with less accuracy than the five-feature logistic regression classifier.
“Such a process could streamline autism diagnosis to enable earlier detection and earlier access to therapy that has the highest impact during earlier windows of social development. Further, this approach could help to reduce the geographic and financial burdens associated with access to diagnostic resources and provide more equal opportunity to underserved populations, including those in developing countries,” Tariq and colleagues wrote.
They added future studies should determine the most viable method of crowdsourcing video acquisition and feature tagging. In addition, trials with larger cohorts at various stages of autism diagnosis and developmental delay spectrums are needed to further explore the home video-machine learning tool’s full potential as a diagnostic tool, they added. – by Janel Miller
Disclosures: Tariq reports no relevant financial disclosures. Please see the study for all other authors’ relevant financial disclosures.
Perspective
Back to Top
Tariq and colleagues’ results are multifaceted but, on the whole, they are not surprising. The nonexpert tagged video-based diagnoses ranged in accuracy from low to high, depending on the statistical model, the age range, and presumably the level of cognitive function of the children participating in the video source material. Given the perfect reliability of diagnostic agreement between experienced clinicians viewing the same very short video source material, the fact that nonexperts also show modest to high degrees of accuracy is not surprising.
There is considerable interest in new methods to improve early screening to make earlier diagnoses and enable earlier intervention, which has been shown to be important for better long-term outcomes. Novel research is critically important as current screening procedures are very inaccurate. Most research in this area moved toward multi-stage screening, with the broadest screening in stage one being universal screening for all children. Given the low incidence of autism in the general population (1% to 2%), current approaches are very inaccurate. Tariq et al’s study design seems like it would correspond to stage 2 screening, where all children have already been identified as at risk for autism. However, their study age range includes many older children who would not ordinarily be in samples testing screening methods.
This group of investigators has taken a novel approach of taking parent-provided home videos of variable duration, but all relatively short, and apparently without any instructions to the parents as to what kind of behaviors to capture. It is important to note that not all the parent-provided videos were acceptable to the researchers (16% were excluded from the analyses) and so the results are only on the remainder.
The 100% diagnostic agreement between trained autism diagnosticians suggests that autism portrayed in the videos was rather frank, and in this regard not representative of the general population of youth with autism. In the general population of youth with autism, experts disagree at least some of the time because the outward expression of autism can be subtle and inconsistent. Even two autism experts watching the same 30-plus minute video of an actual autism diagnostic evaluation never agree 100% of the time and those longer videos provide much more information such that reliability should be greater. Moreover, studies of the reliability of autism between a community practitioner and an autism specialist conducting a more detailed autism evaluation show that there is diagnostic disagreement 10% to 20% of the time. This baseline information on diagnostic agreement in prior research is useful because it appears that the children studied by Tariq et al must exhibit behaviors which are much easier to identify as autistic or not. This puts in context the accuracy found using video tagging with machine learning, as it would be expected these accuracies would also be somewhat higher than might otherwise be found in the course of general clinical practice.
This said, the approach taken by the investigators in this novel and promising. More research is required. It would be premature to suggest that these procedures are ready for routine clinical use by practitioners. While highly novel approaches such as the one being studied here are critically important to move the field forward, much more research needs to be done to know whether these results can and should impact diagnostic practices.