DOs hope that new research will lead to universal use of COMLEX for DOs

New research has found positive correlations between performance on Comprehensive Osteopathic Medical Licensing Examination (COMLEX) and American Board of Surgery In-Training Examination (ABSITE)/American Board of Surgery (ABS) exams. The study’s authors hope that their published research will encourage all surgery residency programs to no longer require osteopathic candidates to take the United States Medical Licensing Examination (USMLE).

The correlation shows program directors that they can use an applicant’s COMLEX performance to predict success on the ABSITE/ABS, similarly to how they would use an MD or IMG applicant’s USMLE scores to do the same. The study notes that ABS pass rates are an indicator of residency program quality, which means program directors are interested in hiring residents who will do well on these exams.

The study was published in the Journal of Surgical Education on Aug. 8. Surgical resident Grace Young, DO, also recently presented the research to the Association of Program Directors in Surgery (APDS).

“This research provides surgery program directors with evidence and understanding of COMLEX-USA scores, so they no longer need to ask DOs to take USMLE,” said Jeanne M. Sandella, DO, lead author of the study and vice president for professional development initiatives and communications at the NBOME. Dr. Sandella is also a member of The DO’s Editorial Advisory Board.

The study authors noted that the extra requirement for DO surgery residency applicants to take the USMLE exam imposes unnecessary financial and stress burdens on DO applicants. To learn more about NBOME’s research and the steps they plan to take to educate the GME community about it, The DO recently talked with Kristen Conrad-Schnetz, DO, one of the study’s authors. Dr. Conrad-Schnetz is also a surgical residency program director and a past president of the American College of Osteopathic Surgeons.

How were these correlations discovered?

We found them when we were doing some initial research, then NBOME leadership reached out to ABS with a summary, and their representative contacted us to get approval and agreements between the two to share data. They then compared applicants’ USMLE scores and COMLEX scores with their later ABSITE and ABS scores. They were able to do a regression model that showed a curve with a percentage of passing quality evaluations.

How long has this research been going on?

This project in particular has been underway since 2023; our original project started in 2021. It took a long time to get data use agreements settled with ABS.

You recently presented this research to the APDS. How did the presentation go?

It went very well. We actually had a surgeon stand up and say they would not find a more robust study with a stronger correlation—it’s answering a question a lot of people have. DOs who have taken both exams received the presentation well and agreed with where the research is going.

How strong were the correlations between COMLEX-USA performance and performance on ABSITE and ABS board exams?

The strongest correlation was with the ABS exam, particularly with ABS QE. They were stronger than similar studies looking at USMLE, and our study used a much larger population of residents and physicians.

How will the evaluation of osteopathic applicants evolve as we move toward no longer requiring USMLE?

The first step is getting open and transparent support of COMLEX from other organizations that care about and look at the recruiting process. Now that the data is published, we can move forward with getting it into the hands of the people who need it so they have the tools they need to make decisions. I believe we’ll have a robust marketing approach, especially with social media, because we can tag the organizations.

COMLEX should be accepted by all, and it’s highly encouraged. For the surgical board to come out and say this, it’s a step in the right direction.

Could there be similar correlations among other specialties?

There are many other articles that look at the relationship between COMLEX scores and performance on residency in-service and board certification examinations, including in family medicine, emergency medicine, pediatrics, internal medicine and OB/GYN. NBOME is working on follow-up research with the American Osteopathic Board of Internal Medicine currently and plan to do more.

Any final thoughts?

The only way to change paths is to give people data and not be afraid to throw ourselves into spaces where people need to see it. We are doing this, and we are moving the ball forward.

Transparency for residents

NBOME strongly encourages all programs to offer transparency in all aspects of their recruiting processes in public-facing policy statements. This may include a statement regarding acceptance and use of both USMLE and COMLEX scores.

The APDS Recruitment Task Force has updated its guidelines for 2024-2025 to emphasize the importance of a holistic review process. These guidelines also stress the need for residency programs to transparently disclose how COMLEX-USA Level 2-CE and USMLE Step 2 CK scores will be evaluated within this holistic framework.

COMLEX-USA percentiles can be calculated from three-digit scores using NBOME’s Percentile Score Converter. This score converter also is accessible on the program director sites for ERAS and Liaison.

Related reading:

How OMM solved a year-long mystery in 15 minutes

Osteopathic innovation and storytelling highlighted in a new platform & new podcasts