Model performs well in predicting suicide attempts across races, ethnicities

A model developed to predict suicide attempts performed well across races and ethnicities, according to a research letter published in JAMA Psychiatry.

“Innovative prevention strategies are needed to reduce U.S. suicide rates, which have been steadily increasing with a recent disproportionate increase among Black and Hispanic populations,” Santiago Papini, PhD, a postdoctoral research fellow in the Kaiser Permanente Division of Research in Oakland, California, and colleagues wrote. “Predictive models of suicide risk have been developed using machine learning with electronic health records.”

Although some models have been successful, some have performed with subpar accuracy among subgroups of the general population, such as among American Indians and Alaskan Natives, the researchers wrote.

Papini and colleagues used an existing logistic regression prediction model of suicide attempts — both fatal and non-fatal — to evaluate EHR data from 254,777 patients aged 12 years or older who had 1,408,682 outpatient mental health encounters at Kaiser Permanente Northern California from October 2016 to September 2017.

Among 1,408 patients who attempted suicide, there were 9,093 visits followed by suicide attempts within 90 days, an attempt rate of 6.5 per 1,000 visits. Attempt rates by race and ethnicity ranged from 4.6 per 1,000 visits among patients with other/unreported race and ethnicity to 19.3 per 1,000 visits among American Indian or Alaskan Native patients.

The overall mean area under the curve was 0.85 (95% CI, 0.84-0.85) with little change across racial and ethnic subgroups.

With a specificity of 95%, sensitivity ranged from 28% (95% CI, 0.22-0.36) among American Indian or Alaskan Native patients to 48% (95% CI, 0.35-0.6) among Native Hawaiian or Pacific Islander patients. With 99% specificity, sensitivity ranged from 7% (95% CI, 0.04-0.1) among patients with other/unreported race and ethnicity to 38% (95% CI, 0.26-0.51) among Native Hawaiian or Pacific Islander patients.

“A limitation of setting thresholds at high specificity is relatively high false-negative rates, which in this study varied across racial and ethnic groups; this underscores the importance of clinical workflows that always consider additional sources of information when assessing suicide risk,” Papino and colleagues wrote. “Ongoing efforts to incorporate social determinants and related factors into the electronic health records of integrated health care delivery systems may also be critical to improving suicide prediction models, particularly among historically underserved groups.”