This article is more than 5 years old. Information may no longer be current.

ABIM IM-MOC examinations concordant with practice

A majority of questions from the American Board of Internal Medicine’s Maintenance of Certification program were concordant with medical conditions seen in internal medicine practices, yet there were some discrepancies, indicating opportunities for improvement, according to research published in JAMA.

“Success on the internal medicine (IM) examination is a central requirement of the American Board of Internal Medicine’s (ABIM’s) Maintenance of Certification program (MOC),” Bradley Gray, PhD, of ABIM, and colleagues wrote.

“Considering that 91% of general internists are certified by the ABIM, measuring the degree to which examination content aligns with clinical practice is both a key element of the validity of the examination results and an important policy question,” they added.

Gray and colleagues sought to determine if the questions on the ABIM IM-MOC examination are concordant with conditions seen in general internal medicine practice. They calculated the percentage of questions for 186 medical condition categories, such as diabetes mellitus, ischemic heart disease and liver disease, from the examination blueprint using content from the 2010 to 2013 IM-MOC examinations. They used data from the 2010 to 2013 National Ambulatory Medical Care Surveys and 2010 National Hospital Discharge Survey, including 13,832 office visits and 108,472 hospital stays, respectively, to compute nationally representative estimates of conditions seen by internists in practice.

A total of 3,600 questions were administered during the 2010 to 2013 IM-MOC examinations. Of those, 3,461 (96.1%) were related to the 186 study conditions (mean, 18.6 questions per condition). The researchers found that 69% of examination questions (95% CI, 67.5-70.6 involving 158 conditions) were concordant with conditions seen in practice. Comparing questions and office visits only revealed that 58.08% of questions (95% CI, 56.43-59.72 of all examination questions) involving 145 conditions were concordant, whereas comparing questions and hospital stays only revealed that 42.07% of questions (95% CI, 40.42-43.71) involving 122 conditions were concordant.

Discordance was seen in 30.97% of questions (n = 1,072; 95% CI, 29.43-32.51 of all examination questions) involving 28 conditions. The researchers noted that concordance could be achieved for these conditions by having either greater or fewer examination questions related to the condition, contingent on the type of discordance.

These findings indicate “that the IM-MOC examination has generally been consistent with the conditions seen in practice. However ... the study also identified potential opportunities for improvement,” Gray and colleagues concluded.

“The ABIM is using data in an effort to ensure that future examinations better reflect the conditions seen by practicing internists... Future research is needed to track whether changes in the IM-MOC examination that were informed by this and other analyses resulted in discernible improvement in the relevance of the IM-MOC examination content,” they added.

In a related editorial, Adam B. Schwartz, MD, from the New York University School of Medicine, and J. Sanford Schwartz, MD, from the Perelman School of Medicine at the University of Pennsylvania, wrote that Gray and colleagues make an important effort to provide evidence on the validity and relevance of the IM-MOC examination in the current investigation.

However, they noted that all authors are employed by or affiliated with the ABIM and therefore, future analysis and replication and extension of the study by independent investigators are essential to confirm the findings.

“The ABIM must strive for continuous improvement and is most relevant and useful when it asks for help and listens to its members, a process that it has been involved with for the past several years,” Schwartz and Schwartz concluded. “Open, ongoing evaluation of the recertification examination will be essential, along with frank and productive discussion about how to ensure the continuous improvement, excellence, and relevance of the recertification process.” – by Alaina Tedesco

Disclosure: Gray and colleagues report receiving support from the ABIM. A. B. Schwartz reports being board certified in internal medicine by the ABIM in 2015 for 10 years and being enrolled in the ABIM’s MOC program currently. J. S. Schwartz reports receiving time-unlimited board certification from the ABIM.