Disparate measures: Study of world’s eye hospitals finds wide variation in outcomes reporting

A Kellogg Eye Center-led study of major eye hospitals around the world reveals vastly different approaches to publicly reported outcomes of eye care, a finding with implications for patients seeking transparency in care and researchers looking to collaborate beyond their institution.

Published in December in BMC Ophthalmology, the report marks the first international study of its kind. Researchers explored eight leading eye hospitals that made their care outcomes publically available in the United States, Great Britain, Australia, Singapore, and India.  The results describe a widely variable landscape when it comes to how each measures and reports outcomes for common eye conditions.

“The first point is that not many places publicize outcomes at all. These are the handful of major eye hospitals that do, and they each do it very differently. There’s not a lot of overlap,” said Jennifer Weizer, MD, Clinical Associate Professor of Ophthalmology and the Director of Quality Improvement at Kellogg Eye Center (KEC). “Three of the hospitals we looked at are in the US, and even they are quite disparate.”

Weizer co-authored the study with KEC colleagues Paul Lee and Joshua Stein, along with former resident Monica Michelotti, the paper’s first author, who is now practicing in Oregon. They worked with overseas collaborators at institutions affiliated with the World Association of Eye Hospitals, an organization that will bring its annual meeting this summer to Michigan Medicine and KEC. The conference – the group’s first in the US – is expected to draw as many as 200 top-level administrators and faculty to Ann Arbor from prominent eye hospitals around the world. Weizer is slated to present the project results at the meeting.

Dr. Jennifer Weizer

Outcomes reporting is a growing field, particularly in the US where the Affordable Care Act now ties reimbursement levels to the public availability of such metrics. Internationally, having institutions similarly measure and report outcomes fosters collaboration and data sharing.

“We expected and certainly found that there is not much of a standard. Everyone is reporting different things, and I was still surprised at how dissimilar the measures were,” said Michelotti, MD, a 2012 UMMS graduate who completed her residency at KEC in 2015. “There’s a debate about exactly what to focus on. But that’s an important conversation to have for both patients and researchers.”

Among other things, the team compared outcomes data for two common ophthalmology conditions – cataract surgery and macular degeneration – and found only limited overlap in outcomes measured for both. And in no cases were hospitals including patient-reported outcomes (i.e., quality-of-life assessments) in their reports. Such metrics are important but also more labor intensive to capture, said Weizer.

“One thing that gets overlooked is the amount of work it takes amass and analyze all of this information, particularly patient survey data. Some of the places that are the best at this have an army of people to do it,” she said. “But increased scrutiny on outcomes is a good thing and it’s not going away. The question is how we can learn from each other so that we are all comparing apples to apples.”