One of the ABA’s goals is to support research that examines the validity and value of obtaining initial board certification and participating in continuing certification to benefit medical education, diplomates and patients.
In 2017, we hosted a Research Summit, which included clinicians as well as experts in medical education, cognitive science and assessment. Deliberations from that meeting helped inform the work of our Board’s Research Committee – which includes six directors and two non-director anesthesiologists, and support from several Ph.D. psychometricians.
Since the summit, eight articles have been published in peer-reviewed journals, including five this year – two related to initial certification, two about continuing certification and one regarding physician wellbeing.
Just this month, Anesthesia & Analgesia published three articles that provide great insight about ABA board certification.
The first study, titled, “The American Board of Anesthesiology’s Staged Examination System and Performance on the Written Certification Examination After Residency,” compared the performance of anesthesiology residency graduates who took the written examinations for board certification before and after the introduction of the BASIC Examination in 2014. The goal of the BASIC Exam, taken at the end of the CA 1 year, was for residents to improve knowledge acquisition during their time in training. I know that every resident wants to become board certified and works hard to attain that achievement. A 2018 study showed that adding the BASIC Exam accelerated the improvement of In-Training Exam performance as residents progressed through the three years of training. However, it was not known whether adding the BASIC Exam had also improved knowledge once residents finished training as measured by the written certification exam taken after training. The November 2019 study used a statistical method to equate test scores using common test items and found that the first two groups of residents that finished residency having taken the BASIC Exam not only scored seven to eight points higher than the prior baseline cohort that had not taken the BASIC Exam, but also had higher pass rates (94% – 96%) than the pre-BASIC Exam resident graduate groups (92% – 93%). This is likely due to residents studying more and earlier during training.
The second noteworthy paper, titled “The American Board of Anesthesiology’s Standardized Oral Examination for Initial Board Certification,” chronicles the history and evolution of the oral examination since its first administration in 1939. Everyone in residency is thinking ahead and knows that to become board certified this exam is required. People often have questions about how the exam is developed or scored. Although 14 of the 24 American Board of Medical Specialties Member Boards use oral examinations as part of the certifying process, this is the first paper to provide detailed descriptions of how the exam is developed, administered and scored.
You may be familiar with Miller’s pyramid of assessing clinical competence. The first tier in the pyramid is “knows,” which is knowledge of facts and is tested by traditional multiple choice questions. The next higher level tier of clinical competency after “knows” is “knows how,” which is assessed by the oral examination. That is why we have an oral exam, to assess how candidates apply their knowledge, judgment and management skills in clinical scenarios. The oral examination measures aspects of clinical performance not accounted for by the written certifying examination, and passing the oral examination is associated with a decreased risk of subsequent license actions against the anesthesiologist. I am really happy this article has published because it will be a great resource for anyone with questions about the exam.
The third important paper is the “Association Between Participation and Performance in MOCA Minute and Actions Against the Medical Licenses of Anesthesiologists.” In 2016, we launched MOCA Minute®, the web-based longitudinal assessment comprised of 120 questions annually. All anesthesiologists certified in 2000 or later who participated in MOCA Minute were included in this research. It turns out that 1.2% (245/20,006) of these anesthesiologists had actions taken against their medical licenses. Reviewing medical license actions is an accepted method used to identify physicians’ performance deficiencies. The study’s major finding was that both timely participation and meeting the MOCA Minute performance standard are associated with a lower likelihood of being disciplined by a state medical board.
While our research has been informative, it is clear that all research has limitations. However, I am very proud of these studies as they inform the ABA about what we need to do to further validate and show the great value of being board certified in anesthesiology.
If you have other certification-related research questions you think deserve attention, let us know.
Excellent comprehensive report on the matter of Board Certification. This is a subject replete with controversies.
I do not support MOCA as it currently exists. I would be more supportive if you would study participation in MOCA as it relates to the Closed Claim Project. Is there a demonstrable correlation between those who are coerced to participate in MOCA and a lower incidence of closed claim incidents? Then, you may have something to point to regarding the utility and value we, the “participants”, derive from the exercise.