Cream of the crop

Best medical schools? Profession mixed on fairness, merits of U.S. News rankings

School representatives agree that the ranking methodology is flawed, but they differ on what should be done about it.

Topics

A few osteopathic medical schools fare well in the latest U.S. News & World Report rankings of “Best Medical Schools.” In primary care, which is considered one of the two major ranking categories, the Michigan State University College of Osteopathic Medicine in East Lansing stands at No. 14 and the University of North Texas Health Science Center Texas College of Osteopathic Medicine (UNTHSC/TCOM) in Fort Worth at No. 20. In addition, the West Virginia School of Osteopathic Medicine (WVSOM) in Lewisburg ranks 9th in rural medicine and 12th in family medicine.

What’s more, three osteopathic medical schools make the Top 20 in geriatrics. UNTHSC/TCOM ranks 16th, while the University of Medicine and Dentistry-School of Osteopathic Medicine (UMDNJ-SOM) in Stratford ties for 19th place with the University of New England College of Osteopathic Medicine in Biddeford, Maine.

Many other osteopathic medical schools, however, are poorly ranked or unranked, and medical school rankings can vary mysteriously from year to year.

A number of osteopathic medical school representatives agree that the U.S. News ranking methodology is flawed, but they differ as to what should be done as a consequence. School administrators have varying views on whether the ranking system is a challenge to surmount or a travesty to shun.

Compounding the problem, U.S. News rankings are sometimes taken out of context by organizations that adapt the statistics for their own purposes. The popular physician rating site Vitals.com, for example, draws from the rankings to assign a quality score to medical schools. Vitals gives the vast majority of osteopathic medical schools just one out of four stars.

Call to boycott?

Joining five other osteopathic medical schools, the Ohio University Heritage College of Osteopathic Medicine (OU-HCOM) in Athens opted out last year from participating in U.S. News’ annual data and opinion surveys, which form the basis of the med school rankings. OU-HCOM’s director of institutional assessment and planning, Alex Westerfelt, PhD, hopes to spur other colleges in the profession to discontinue participation in the U.S. News ranking system, which he asserts is subjective, unscientific and strongly biased against osteopathic medical education, despite the high rankings of a handful of DO schools.

“The rankings are a very successful marketing effort but are not a valid research effort,” Dr. Westerfelt contends. “We have to stop letting a business venture define quality in medical schools.”

Dr. Westerfelt takes particular issue with the primary care rankings, for which, he insists, U.S. News “uses inferior methodology that no peer-reviewed journal would ever consider.” Given their strength in producing primary care physicians, osteopathic medical schools have been woefully underrepresented among the top-ranked institutions, he says.

[story-sidebar id=”182189″]

The U.S. News ranking system gives each medical school a composite score of up to 100 points based on four broad components:

  • Quality assessment, weighted at 40%.
  • Proportion of graduates entering primary care, 30%.
  • Ratio of full-time faculty members to students, 15%.
  • Selectivity of admissions, 15%.

To come up with its quality assessment, U.S. News sends a survey to medical school deans, associate deans for academic affairs, and heads of internal medicine departments, asking them to rate each of approximately 160 allopathic and osteopathic medical schools on a scale of 1 (marginal) to 5 (outstanding), with “don’t know” also available as an option. In addition, the publisher sends a similar survey to a sample of MD residency directors in internal medicine, family medicine and pediatrics. Because osteopathic medical schools are much fewer in number and less well-known than allopathic medical schools, especially in regions of the country that do not have an osteopathic medical college or many practicing DOs, osteopathic medical education faces an unfair disadvantage in these opinion surveys, Dr. Westerfelt says.

“Both pools of evaluators on average rate the quality of osteopathic medical schools lower than that of allopathic medical schools,” Dr. Westerfelt says. “Those opinions count for the bulk of a school’s score and subsequent ranking.”

To calculate the percentage of a medical school’s graduates going into primary care, U.S. News uses the number of graduates entering residencies in family medicine, internal medicine and pediatrics. Although U.S News reports in a separate list that the Top 5 producers of primary care residents are osteopathic medical schools, even this component of the ranking system doesn’t give osteopathic medical education its expected edge because the 30% weight is not sufficient to compensate for the more heavily weighted quality assessment, Dr. Westerfelt says.

Moreover, the primary care proportion fails to present an accurate picture of those graduates who will actually practice as primary care physicians after serving their residencies, according to Dr. Westerfelt. The ranking system does not take into account that many residents in internal medicine go on to subspecialize. The fact that far more MD graduates than DO graduates pursue internal medicine, as opposed to family medicine, residencies suggests that the U.S. News primary care percentages for leading allopathic medical schools are inflated, he points out.

The student selectivity component of the U.S. News primary care formula weights average scores on the Medical College Admission Test at 9.75%, mean grade point average at 4.5%, and average acceptance rate at 0.75%. Osteopathic medical schools have traditionally admitted students with somewhat lower MCAT scores and GPAs than do allopathic medical schools, so it is not surprising that DO schools lose further ground with this portion of the ranking formula, according to Dr. Westerfelt.

“Research has shown that the higher the MCAT scores, the more likely students will go into subspecialties,” notes Bruce D. Dubin, DO, JD, the dean of the Rocky Vista University College of Osteopathic Medicine in Parker, Colo. Osteopathic medical schools place more emphasis on candidates’ expressed commitment to primary care and to osteopathic principles and practice, Dr. Dubin says.

Osteopathic medical colleges also come up short on faculty-student ratio because DO schools use a higher proportion of part-time faculty members, who are not counted in this ratio.

By participating in this ranking system, “osteopathic medical schools have abdicated the right to define quality,” Dr. Westerfelt argues. “The U.S. News ratings are determined by uninformed opinions and by inputs, such as MCAT scores and GPAs, rather than by outputs—the quality and the number of our graduates who become primary care physicians.”

The president of the American Association of Colleges of Osteopathic Medicine (AACOM), Stephen C. Shannon, DO, MPH, agrees that the U.S. News rankings “give biased and erroneous information.” He notes that AACOM’s member colleges have discussed refusing to cooperate with the publisher’s annual surveys as a group, but at this time the majority of osteopathic medical schools consider participating in the country’s most visible medical school ranking system to be in their best interests. Nevertheless, the debate continues as to whether they should continue to participate.

Should osteopathic medical colleges ever decide collectively to eschew the U.S. News rankings, they would not be the first professional group to do so. In 1994, all U.S. dental schools stopped participating in the dental program rankings. And four years ago, dozens of independent liberal arts colleges ceased participation in the reputation survey portion of the U.S. News “Best Colleges” rankings.

Potential alternatives

Like other well-rated osteopathic medical schools, WVSOM touts its high U.S. News rankings on its website. “Our potential applicants look at these rankings,” says the school’s president, Michael Adelman, DO, JD. Although he is proud that WVSOM is recognized for excellence in rural medicine and family medicine, he does question the methodology used for primary care.

“It is ironic that we are in the Top 10 in rural medicine and the Top 15 in family medicine, but we’ve fallen to 94th in primary care,” Dr. Adelman observes. “That doesn’t make sense to me.”

Dr. Adelman would prefer to see a ranking system for primary care that assesses how well a medical school fulfills its mission. If a school’s mission is to provide primary care physicians to underserved areas, then the rating should be based on the number and quality of graduates practicing in primary care specialties in such areas, he says.

Dr. Shannon agrees that a better gauge of a medical college’s caliber would be the extent to which the school carries out its mission. An article in the June 2010 issue of Annals of Internal Medicine proposes an alternative ranking system based on a “social mission score” that reflects the percentages of graduates who are primary care physicians, who practice in health professional shortage areas, and who are underrepresented minorities. Such a system merits further consideration, Dr. Shannon believes.

Osteopathic medical schools, in fact, do not rank highly overall in the system described in Annals because of the weight accorded underrepresented minority enrollment. “Our schools overall have had difficulty recruiting minority students,” Dr. Shannon says.

The most objective educational ranking system Dr. Westerfelt has seen is a collaboration of the National Academies and the National Research Council to evaluate and rank PhD programs. The resulting document is more than 200 pages long—more than most prospective students will wade through, he concedes. However, an independent organization, PhDs.org, has adapted this dense report into a more accessible and flexible online format that allows users to rank programs according to their own priorities.

Public relations value

The director of communications for the Lake Erie College of Osteopathic Medicine (LECOM) in Erie, Pa., Pierre Bellicini agrees that the U.S. News ranking system for medical schools is flawed. But he notes that after much effort on the part of LECOM and some other osteopathic medical schools, the publisher has been willing to adjust its surveys over the years. For example, when U.S. News first published its medical school rankings, institutions were ranked almost entirely according to their strength in research. Although U.S. News still lists “Best Research” medical schools, among which osteopathic medical colleges as a group rank poorly, the addition of primary care rankings and specialty rankings marks a change that has enhanced the visibility and recognition of osteopathic medical education, Bellicini says.

“Being among U.S. News’ Top 50 in primary care is one of the few ways for osteopathic medical schools to get noticed nationally,” Bellicini notes. The media attention garnered by highly ranked DO schools benefits the entire profession, he says.

Bellicini points out that medical school applicants today look at several other sources of information besides the U.S. News rankings, including school websites and Facebook pages, so osteopathic medical schools needn’t be overly concerned about low rankings.

No perfect methodology

“All in all, I think that the rating system U.S. News uses is good, but there is definitely room for improvement,” says Thomas A. Cavalieri, DO, the dean of UMDNJ-SOM. “It is difficult to come up with any evaluation system that would meet everyone’s needs. There is no perfect way to compare schools.”

Osteopathic medical schools are at a disadvantage because respondents to the U.S. News primary care survey “simply aren’t aware of our colleges,” Dr. Cavalieri notes. This can be ameliorated by including family medicine, as well as internal medicine, chairmen in the survey, he suggests. “If U.S. News would send the surveys to chairs of family medicine, osteopathic medical schools would do much better,” Dr. Cavalieri says. “The chairs of family medicine at allopathic medical schools are very aware of the strengths of osteopathic medical schools in family medicine.”

“We stand behind the validity of our research,” says Robert J. Morse, the director of data research for U.S. News & World Report, while noting that he is open to suggestions such as Dr. Cavalieri’s. The technique of surveying a large number of people on their opinions and quantifying the results is common research practice, he says.

Dr. Cavalieri asserts that the osteopathic medical profession needs to do a better job of promoting itself, both to the broader medical community and the public. Greater visibility would improve DO schools’ standings in the rankings, he says.

Because of the osteopathic medical profession’s long struggle for parity with allopathic medicine, Dr. Cavalieri believes it would be detrimental for the profession to separate itself from the country’s best-known ranking system for medical schools.

One alternative source to the U.S. News rankings, The Princeton Review’s book Best 168 Medical Schools, lists osteopathic medical schools in a separate section titled “Osteopathic Profiles” and encourages nontraditional students to consider osteopathic medicine. In Dr. Cavalieri’s view, this is far worse for colleges of osteopathic medicine than the U.S. News rankings, especially given that the “Osteopathic Profiles” follow the “Naturopathic Profiles.”

Long-debated issue

Dr. Westerfelt remains convinced that the U.S. News rankings only harm the osteopathic medical profession. He plans to write a letter in the fall—when U.S. News begins collecting data for the next ranking list—to encourage osteopathic medical school administrators to stop participating in the annual surveys. “This is a race we shouldn’t even waste time trying to win,” he says.

But many allopathic medical schools also feel this way and have so for years. “The annual U.S. News & World Report rankings of U.S. medical schools are ill-conceived; are unscientific; are conducted poorly; ignore the value of school accreditation; judge medical school quality from a narrow, elitist perspective; do not consider social and professional outcomes in program quality calculations; and fail to meet basic standards of journalist ethics,” stated an article in the October 2001 issue of Academic Medicine, the peer-reviewed journal of the Association of American Medical Colleges. “The U.S. medical education community, higher education scholars, the journalism profession, and the public should ignore this annual marketing shell game.”

10 comments

  1. Stephen Blythe, D.O.

    Since an internal medicine residency is often the gateway to a medicine subspecialty, the number of graduates entering an IM residency hardly reflects a colleges primary care orientation. As stated, looking at the percentage of graduates practicing in primary care would be far more valuable. And there is still the allopathic bias against medical education taking place at the community hospital level. My internship at a very busy community hospital put me at the front line of patient care and procedures. For example: While I saw no advanced limb salvage procedures during my ortho rotation, I spent the month working alongside three very busy orthopedists – as their only intern – examining knees, elbows, necks, and backs, reviewing x-rays, applying splints and casts – from morning to night. I would put my basic primary care orthopedics training up against any Mayo Clinic intern any day.

  2. Joe Baber, DO

    Osteopathic schools must take a careful look at the materials and methods used to make these assessements. Even if these results would not stand up against the same scrutinity that physicians use to critique primary literature, it should be openly discussed. While we know that NEJM publishes many industry sponsored studies infused with bias, they still generate national attention and turn the hands and pens of practicing and prescribing physicians. Instead of disregarding the data, the osteopathic profession should critique it in a way to formulate a self assessement for continued self-improvement. From my experience in a leadership position in an ACGME IM residency program, there remains a wide range of clinical skills proficiency amongst our osteopathic incoming residents. I have concerns that as the profession rapidly responds to meet the needs of the community in underserved areas with newer medical schools, the oversight and quality assurance of the clinical training years must be more closely assured so that all graduates will remain to be optimally equipped to enter any training specialty.

  3. Pouya Bahrami DO

    Osteopathic Medical Schools need to be more independent minded and innovative. We have traditionally waited for “others” to define us before we fight back. I find it interesting that there is not a single “fair” proposal by Osteopathic institutions to rank medical schools. Why not create a fair methodology and rank all schools in the Osteopathic and Allopathic disciplines? Such a fair, and balanced ranking, published by the Osteopathic profession, could have subcategories that include primary care, specialty, or underserved champions. Once we, the Osteopaths, begin to rank schools in a methodology that most fair minded institutions and scientist would agree with, not only we attract the attention of the academia, but we also can fairly see how we are doing and what we should be doing better. We can lead by example and not worry about the US News. How about that for a change?

  4. Jake

    After reading this article, I feel terrible on choosing osteopathic school over allopathic school. I declined two allopathic acceptances to go to a osteopathic school that I truly loved, but after reading this article, I wish I hadnt declined my allopathic acceptances.

  5. Robert C. Bowman, M.D.

    To “measure” primary care, specific primary care delivery measures are needed. Primary care rankings can be established that adjust for primary care retention, volume, years in a career, and % active. The Standard Primary Care Year (SPCYr) is such a measure – specific to class year of graduation and reflecting the future primary care delivery throughout a career.

    A perfect score of 35 SPCYrs per graduate is the result of 100% PC retention with 100% volume with 100% active for 35 years. Family medicine residency graduates contribute 22 – 28 SPCYrs during their careers mainly due to over 80% primary care retention.

    The most elite medical schools using US News Rankings, MCATs of matriculants, or NIH Research dollars trail with least primary care delivery per graduate at 1 – 3 SPCYrs per graduate – the same level as nurse practitioners not training in family practice.

    Osteopathic schools in the 1960s averaged 20 Standard Primary Care years per graduate due to over 65% of graduates in family practice but have decreased to about 8 on average (17% family practice) with a wide range from 6 to 13 SPCYrs per graduate.

    US MD is down to about 4 to 5 SPCYrs per grad or about the same as PA or NP or US MD internal medicine.

    Steady declines in primary care retention defeat primary care delivery in all of the flexible primary care training sources (IM, PD, MPD, NP, and PA). Steady departures from primary care during training, at graduation, and each year after graduation are the result of flexible designs impacted by US policy that neglects primary care (ever more evident with MedPAC recs, Medicare and Medicaid cuts).

    This results in primary care delivery for a school or school type or program more dependent upon the proportion entering family medicine – the remaining permanent PC source most resistant to aberrant US designs after graduation (but most vulnerable at the critical stage of medical student choice).

    US primary care delivery capacity involving 28,000 annual NP, IM, PA, MPD, PD, and FM graduates is limited due to declines to 7 SPCYrs per graduate as only 30% serve in primary care. This is down from 18 – 20 SPCYrs per graduate for the 1980 annual graduates that were half as many at 14,000 but provided 280,000 SPCYrs for the class entering the workforce in 1980 as compared to the more recent 210,000. Twice the annual graduates with one-third of the primary care delivery per graduate indicates failures in primary care training, failures in primary care policy, and the failure of generic expansion. Specific solutions are required with training, spending, and career choice to result in increases in primary care measurements and rankings.

    Bob Bowman ATSU SOMA

  6. Jaffer Dina

    Hi everyone,
    In order to preserve the accreditation/reputation of UMDNJ-SOM:

    1. Please sign the following petition to prevent a merger of UMDNJ-SOM with Rowan and to instead merge it with Rutgers.
    http://www.change.org/petitions/the-governor-of-nj-amend-senate-bill-s2063-to-merge-umdnj-som-with-rutgers

    2. If you are from NJ, please call your local legislators (and let em know that this merger is bad for SOM) and get other physicians involved as well.

    Thanking in anticipation,
    Jaffer Dina

Leave a comment Please see our comment policy