Pass/fail COMLEX: Considerations and outlook from a student’s perspective

On December 17, 2020, the National Board of Osteopathic Medical Examiners (NBOME) made a significant announcement. Level 1 of the COMLEX board series would transition from a three-digit score to a simple Pass/Fail assessment beginning in May of 2022. At the time of the announcement, I distinctly remember doing some quick finger counting. As a member of the class of 2024, I realized I would be one of the first students to not receive a three-digit score for my first board exams.

This decision spurred a lot of speculation in student spaces. Change, even positive change, can be difficult to swallow and hard to comprehend. The first level of the COMLEX has, historically, been a constant pressure and force driving so much of the preclinical years of medical students across the country. By now, the first class of students to experience the Pass/Fail system have largely completed their exams. This makes for a good moment to reflect and provide some further speculation.

This is a complicated issue. No one will fully know the effects of pass/fail boards on osteopathic students until March 2024. In attempting to summarize some of the important pros and cons of this decision, I am drawing on my personal experience and the experiences and opinions of my peers, as well as some of the data available in the program director surveys provided by the National Residency Matching Program (NRMP).

Speaking as a student with nothing more than “Pass” on my board score results who is hoping to match into a competitive surgical specialty, I hold that this is overall a positive change for us. There are negatives, but I believe they are outweighed by the positives.

Pros of pass/fail

One of the more positive points to consider here is the increased opportunity and latitude given to our preclinical students because of the time they now have that they aren’t spending trying to stress and strain for every possible point. It was accepted practice before the change for students to begin studying for the boards at the beginning of their second year, if not at the moment of matriculation. The students in the class above us recommended at least four to five hours of studying for boards per week for the first semester of the second year.

I didn’t start board studying until my COM provided a dedicated period. This gave me four to five extra hours every week that I could put toward my professional development. I was able to participate in so many more extracurriculars and leadership positions than I otherwise would have been able to. The positive impact this had on me, and my career, cannot be overstated.

Another positive change I anticipate will be a result of this change may come from our institutions. With less pressure and focus placed on the first level of boards, there is an opportunity for our institutions to be more creative and liberal with our preclinical curriculum. There is an opportunity to add more clinical experience, leadership training and humanities instruction.

It could truly be an opportunity for us to become more than just an encyclopedia of basic facts. I hope that we as a community will seize this time to forge ourselves into the complete physicians that we will need to be in our increasingly complicated health system.

The final positive that I want to highlight comes from the NRMP program director survey. I’m comparing the responses from general surgery program directors to the survey from 2018 to 2021. While programs still have access to board scores at this point, they must be preparing for this shift, and we can see that through some of the responses to the survey.

The 2021 survey shows a more holistic view of applicants in terms of choosing who to interview and who to rank. One example is that in 2018, 81% of respondents listed Level 1 scores as a factor for deciding who to rank, with a weight of 4.4/5. In 2021, only 67% listed this as a factor, with a weight of 4/5.

Perhaps even more surprising, the number of program directors prioritizing Level 2 scores also declined over the same period. In 2018, 78% of respondents listed Level 2 scores as a factor in ranking applicants, with a weight of 4.2/5. In 2021, it was 65% with a weight of 4/5. One of the main concerns at the time of this announcement was that Level 1 would simply be replaced by Level 2. It remains to be seen what happens during the Match in 2024, but it may not be as simple as we had originally thought.

Cons of pass/fail

There are some negatives that I have observed that may have a noticeable effect on students, but I believe many of these concerns will be improved over time with increasing institutional memory and wisdom. Some students suffered from a lack of vision during the second year. Without such a large target to focus on, students were required to be more creative with how they spent their time to have an optimal outcome, and this was challenging for some.

We’ll need to wait for official data from the NBOME, but more students than usual may be underprepared for the exams, given the need to simply pass. Another important point to mention is the case of the high-performing student. Some students are likely going to perform highly in many aspects of their education, including their board exams. Taking away a board exam score may be a negative for these students, as they likely would have had an impressive score to add to their already stellar CV.

Another con: The first level of the board exams was a way for students to self-select specialty choices to some degree. Now, a student may believe they are competitive for a difficult-to-match specialty all through their third year, only to get a disappointing score on Level 2 and be forced to make a difficult decision later than previously required.

Future outlook and possible implications

Ultimately, I think this is a positive change for the DO community. However, time will tell exactly how this will affect us moving forward in terms of match success for DOs. Best wishes to my peers in the class of 2024 as we trailblaze this path.

Related reading:

Eliminating numeric scores for COMLEX-USA Level 1

The NRMP 2023 Match and ERAS timeline for residency applicants