Computerized adaptive assessment tools valid for veterans with PTSD
Computerized adaptive assessment tools for veterans with PTSD produced valid screening diagnoses and severity scores, according to study results published in JAMA Network Open.
“One alternative to administering traditional assessment measures is computerized adaptive testing (CAT) in which a person’s initial item responses are used to determine a provisional estimate of their standing on the measured trait, which is then used for the selection of subsequent items, thereby increasing the precision of measurement and accuracy of diagnostic screening and minimizing clinician and patient burden,” Lisa A. Brenner, PhD, of the VA Rocky Mountain Mental Illness Research, Education and Clinical Center at the Rocky Mountain Regional Veterans Affairs Medical Center in Colorado, and colleagues wrote. “For complex disorders, such as PTSD, in which items are selected from distinct yet related subdomains (eg, exposure, negative alteration in mood and/or cognition, alteration in arousal and/or activity, avoidance, and intrusion), selection of items is based on multidimensional rather than unidimensional item response theory. Adaptive diagnosis and measurement are fundamentally different.”
According to the researchers, measurement approaches such as CAT aim to move the times to the patient’s severity level; however, in computerized adaptive diagnosis (CAD), the goal is to move the items at the tipping point between a positive and negative diagnosis. Thus, both methods are adaptive but incorporate different statistical approaches.
Brenner and colleagues sought to develop and validate a CAD screener, as well as a CAT, for assessing PTSD symptom severity among 713 U.S. military veterans (585 men; mean age, 52.8 years) at a Veterans Health Administration facility between April 25, 2017, and Nov. 10, 2019. Participants answered a PTSD symptoms questionnaire from the item bank and gave responses on the PTSD Checklist for DSM-5. The researchers interviewed a subsample of 304 participants via the Clinician-Administered Scale for PTSD for DSM-5.
Results showed high sensitivity and specificity per an area under the curve of 0.91 (95% CI, 0.87-0.95) for the CAD-PTSDs’ reproduction of the Clinician-Administered Scale for PTSD for DSM-5 PTSD’s diagnosis. Brenner and colleagues noted that the CAT-PTSD exhibited convergent validity with the PCL-5, as well as tracked PTSD diagnosis, with an area under the curve of 0.85 (95% CI, 0.79-0.89). Further, they reported a correlation of r = 0.95 for the CAT-PTSD’s reproduction of the final 203-itme bank score, with a mean of only 10 adaptively administered items. This represented a 95% reduction in patient burden.
“The findings of this study suggest that, among veterans, the CAD-PTSD and CAT-PTSD appear to provide valid diagnostic screening and dimensional severity scores, with substantial reductions in patient and clinician burden,” Brenner and colleagues wrote. “These measures are scalable and can be integrated into electronic health record systems for routine use in health care settings.”