ProPublica (@ProPublica)is an independent, non-profit journalism newsroom seeking to expose abuses of power and betrayals of the public trust by government, business and other institutions through investigative journalism. Founders and editors include former Wall Street Journal, New York Times and other mainstream media veterans and leaders.
In July of this year, ProPublica caused a bit of a maelstrom with its release of a Surgeon Scorecard, a searchable database of 16,827 surgeons performing one of eight elective procedures in Medicare and their individual complication rates. The organization focused on elective cases because they typically involve healthier patients with the best odds of a smooth recovery. ProPublica notes that previously, patients had little to no information with which to choose a surgeon. Based on the many stories from patients with poor experiences and outcomes from surgery, ProPublica decided to make the complication rates of nearly 17,000 surgeons nationwide public. The database is designed be used by patients to find out more about a surgeon before an operation, and ostensibly by doctors to see their ratings relative to their peers
The elective surgeries measured were:
- Knee Replacement
- Hip Replacement
- Gallbladder Removal, Laparoscopic
- Lumbar Spinal Fusion, Posterior Column & Approach
- Lumbar Spinal Fusion – Anterior Column, Posterior Approach
- Prostate Resection
- Prostate Removal
- Cervical (Neck) Spinal Fusion
Key Findings from the Database
The investigation of five years’ worth of Medicare billing records from 2009 to 2013 indicated that 63,000 Medicare patients suffered serious harm and 3,405 died after procedures widely seen as straightforward and low risk operations.
The investigations also found that many hospitals don’t track the complication rates of individual surgeons and use that data to force improvements; neither does the government.
About 11% of doctors accounted for 25% of the complications. Hundreds of surgeons across the US had rates double and triple the national average. Surgeons with the highest complication rates in ProPublica’s analysis are performing operations in hospitals nationwide daily. The ultimate goal of ProPublica’s investigation is to develop a culture of transparency in healthcare and open discussion of mistakes.
Television and other media outlets have run recent stories about ProPublica’s Scorecard, relating experiences of patients and relatives of patients who had complications following surgery, and noting the lack of information available to patients before an operation. The growing media exposure of the Surgeon Scorecard has also spurred much criticism from surgeons around the country.
Commentary and Criticism
While many critics laud ProPublica’s intentions of transparency in healthcare, concerns regarding methodology and the validity of their ratings system are many. One is that using only data on Medicare patients skews results – that those patients may tend to be older and sicker with possible co-existing health problems that can exacerbate complications. And that not looking at a surgeon’s overall rate, including that of privately insured patients, can be misleading. Here is a rundown of o
Last week Sarianne Gruber (@SubtleImpact) who reports and blogs for us on RCM Answers, did a piece on the scorecard reflecting mixed reviews and providing backdrop on how the data was collected. Read her article, ProPublica’s Surgeon Scorecard has Doctors, Statisticians and Journalists Weighing In.
In an article in Forbes (@Forbes), Dr. Ford Vox claims that ProPublica reporters do not necessarily understand all aspects of an individual case and can end up “steamrolling the reputations of good doctors.” (Forbes April 3, 2014 ProPublica Goes After The Wrong Man In Its ‘Dollars For Docs’ Crusade)
In another view, Dr. John M. Mandrola believes the data are incomplete – that ProPublica used only two surgical outcomes, death or readmissions, without regard to crucial details. He says “billing data aren’t nuanced enough to tease out these crucial details, and that the rates don’t account for sicker patients. In terms of accuracy, Mandrola noted that the scorecard listed surgeons at his hospital under the wrong category, i.e., listing a cardiologist performing knee replacements. In short, bad data is worse than no data. (Medscape (@Medscape) July 23, 2015 Failing Grade for ProPublica’s Surgeon Scorecard)
On the web site KevinMD.com, Dr. Jeffrey Parks goes as far as to say that the Surgeon Scorecard is a journalistic low point for ProPublica. He says, “it distorts reality, clouds data, confuses patients and proffers no insight in how a surgeon might improve his/her results.” (KevinMD (@KevinMD) July 22, 2015 Why the Surgeon Scorecard is a journalistic low point for ProPublica)
Other opinions are more positive as illustrated by these quotes from ProPublica on the scorecard: Dr. Robert Wachter, Professor at University of California, Department of Medicine, says “ProPublica’s Surgeon Scorecard is a courageous database that identifies differences among physicians…in addition to helping consumers, empowers hospital boards of directors.” Dr. Thomas Lee, Professor at the Harvard School of Public Health, says “I think the methodology was rigorous and conservative. I would be surprised if any experienced clinician challenged the basic finding, that there is real variation among surgeons.”
Finally, in a recent NBC Today Show segment (@Todayshow) ProPublica reporter Marshall Allen responded to criticism of the Surgeon Scorecard saying “the numbers are part of the story. Really, this is a starting point for patients to talk to their doctors, talk to their hospitals about these complications.
You can follow the continuing conversation on Twitter at hashtag #SurgeonScorecard.
This article was contributed by Patty O’Toole. Patty has over 30 years of experience in marketing and media research spanning a variety of business including health care, technology, professional services and more. A former New Yorker, she now resides in Phoenix and enjoys music, reading and searching for the perfect pizza.