Combining clinical, algorithmic risk estimates can improve psychosis transition prediction

Sequential integration of risk estimates by algorithms and clinicians can broaden the spectrum of risk prediction for psychosis transition, according to results of a multisite, longitudinal prognostic study published in JAMA Psychiatry.

“The clinical high-risk (CHR) criteria for psychosis have been established to detect vulnerable individuals as early as possible to intercept disease development,” Nikolaos Koutsouleris, MD, of the department of psychiatry and psychotherapy Ludwig-Maximilian-University in Germany, and colleagues wrote. “These criteria identify a patient population with increased incidence compared with the general population, yet only 22% of patients with CHR as detected by ultra–high-risk criteria show a psychosis transition during a 3-year period. The clinical utility of the CHR designation may be further limited because its ascertainment is laborious and confined to specialized, well-equipped health care services that do not sufficiently cover the vulnerable population.”

Despite the development of diverse models to predict psychosis among patients with CHR states, it remains unclear whether efficiently combining clinical and biological models and broadening the risk spectrum to young patients with depressive syndromes can improve prediction. Koutsouleris and colleagues conducted the current study in seven academic early recognition services in five European countries. They analyzed data of a referred sample of 167 individuals with CHR syndromes and 167 with recent-onset depression. A total of 26 developed psychosis; 246 had 18-month follow-up data, which were used for model training and leave-one-site-out cross-validation; 88 patients with non-transition provided validation of model specificity; and 334 healthy volunteers provided a normative sample for prognostic signature evaluation. Further, three independent Swiss projects contributed 45 cases with psychosis transition and 600 with nontransition for external validation of clinical-neurocognitive, structural MRI-based and combined models. Accuracy and generalizability of prognostic systems served as the main outcomes and measures.

The investigators included in the analysis 668 individuals, of whom 334 were controls. Clinicians’ balanced accuracy was 73.2%, with a specificity of 84.9% and a sensitivity of 61.5%. Algorithms exhibited high sensitivity, ranging from 76% to 88%, but low specificity, ranging from 53.5% to 66.8%. A cybernetic risk calculator that combined algorithmic and human components had a psychosis prediction balanced accuracy of 85.5%, with a sensitivity of 84.6% and a specificity of 86.4%. An optimal prognostic workflow exhibited a balanced accuracy of 85.9%, with a sensitivity of 84.6% and a specificity of 87.3%, and had a much lower diagnostic burden by sequential integration of clinical-neurocognitive, expert-based, polygenic risk score-based and structural MRI-based risk estimates as needed for the given patient. Good external validation results supported the findings.

“Our study showed for the first time, to our knowledge, that the augmentation of human prognostic abilities with algorithmic pattern recognition improves prognostic accuracy to margins that likely justify the clinical implementation of cybernetic decision-support tools,” Koutsouleris and colleagues wrote. “New international collaborations, such as the [Harmonization of At Risk Multisite Observational Networks for Youth] initiative, may help to propel a reciprocal and iterative process of clinical validation and refinement of these prognostic tools in real-world early recognition services.”