Do online doctor ratings reflect public perceptions of doctor quality?

When it comes to selecting doctors in this country, it’s almost like searching for a rare collectible on eBay – you need the knowledge to spot the mint condition item out of a group of seemingly similar candidates, and you need some luck. The only trick is that the “mint condition” of a doctor is hard to define. In China, where I come from, we go to a hospital to see doctors. Hospitals are graded and, presumably, better hospitals employ better doctors. In the U.S., when most doctors have the required credential to practice, how do you find a doctor that you can rely on and trust? Word of mouth! You might say. And, in the digital era, online reviews and ratings!

Are online reviews about doctors informative and trustworthy? Do they reflect quality of a doctor? If you have talked to any doctor friends about online reviews, 9 out of 10 times they will tell you that those who post reviews online are the disgruntled patients, so you shouldn’t trust those reviews. However, are these anecdotal stories representative of the wider pattern? Is there any relationship between online doctor ratings and public perceptions of doctor quality? A group of Health IT researchers affiliated with the CHIDS of the Smith School of Business examined exactly this question.

In their study “Vocal minority and silent majority: how do online ratings reflect population perceptions of quality” published in MIS Quarterly, Gao, Greenwood, Agarwal and McCullough examine 1,425 general practitioners in Denver, Kansas City, and Memphis, 794 of whom have been rated online. They use data from the consumer advocacy group Consumers’ Checkbook to measure patients’ underlying offline perceptions of physician quality and use data from RateMD.com to measure physicians’ online ratings.

Their research reveals three notable findings. First, physicians with low patient-perceived quality are less likely to be rated online. In other words, as physician quality increases there is a corresponding increase in the probability of the physician receiving an online rating. Not only that higher-quality physicians are more likely to be rated online, but their second finding also suggests that these physicians tend to receive higher ratings as well. This indicates that online ratings are informative to patients and do reflect population opinions of physician quality. Their third finding reveals a very interesting phenomenon. While higher online ratings reflect higher quality, online ratings are most effective at distinguishing physicians with average quality; they are less sensitive to physicians with low quality, and have no ability to distinguish quality variation of high-quality physicians.

What’s the major take-away for those of us who want to use online ratings to select doctors? First, both doctors and we as patients can rest assured that online ratings are not just composed by unhappy consumers of healthcare. Second, if a doctor does not have online ratings, you can infer that this doctor’s quality tends to be low. Third, higher online ratings reflect higher quality, although such correlation gets weaker for doctors with high ratings. Lastly, we shouldn’t be too picky about minor differences in online ratings among high-end physicians. For example, a rating of 4.9 vs. 5.0 might not reflect meaningful quality difference.

Cite:

Guodong Gao, Brad Greenwood, Ritu Agarwal, and Jeffrey S. McCullough “Vocal Minority and Silent Majority: How Do Online Ratings Reflect Population Perceptions of Quality?” Management Information Systems Quarterly. Vol 39, no. 3 (2015): 565-589

about the author

Yixin Qiu

Yixin Qiu is a Project Manager with Qlarant and is involved in anything related to our Predictive Modeling Solution, PLATO™. Yixin works with developers, modelers, SMEs, as well as outreach and support to deliver a high quality software platform with end users in mind. See all posts from Yixin Qiu.