Impact of public release of performance data on the behaviour of healthcare consumers and providers
- PMID: 30188566
- PMCID: PMC6513271
- DOI: 10.1002/14651858.CD004538.pub3
Impact of public release of performance data on the behaviour of healthcare consumers and providers
Abstract
Background: It is becoming increasingly common to publish information about the quality and performance of healthcare organisations and individual professionals. However, we do not know how this information is used, or the extent to which such reporting leads to quality improvement by changing the behaviour of healthcare consumers, providers, and purchasers.
Objectives: To estimate the effects of public release of performance data, from any source, on changing the healthcare utilisation behaviour of healthcare consumers, providers (professionals and organisations), and purchasers of care. In addition, we sought to estimate the effects on healthcare provider performance, patient outcomes, and staff morale.
Search methods: We searched CENTRAL, MEDLINE, Embase, and two trials registers on 26 June 2017. We checked reference lists of all included studies to identify additional studies.
Selection criteria: We searched for randomised or non-randomised trials, interrupted time series, and controlled before-after studies of the effects of publicly releasing data regarding any aspect of the performance of healthcare organisations or professionals. Each study had to report at least one main outcome related to selecting or changing care.
Data collection and analysis: Two review authors independently screened studies for eligibility and extracted data. For each study, we extracted data about the target groups (healthcare consumers, healthcare providers, and healthcare purchasers), performance data, main outcomes (choice of healthcare provider, and improvement by means of changes in care), and other outcomes (awareness, attitude, knowledge of performance data, and costs). Given the substantial degree of clinical and methodological heterogeneity between the studies, we presented the findings for each policy in a structured format, but did not undertake a meta-analysis.
Main results: We included 12 studies that analysed data from more than 7570 providers (e.g. professionals and organisations), and a further 3,333,386 clinical encounters (e.g. patient referrals, prescriptions). We included four cluster-randomised trials, one cluster-non-randomised trial, six interrupted time series studies, and one controlled before-after study. Eight studies were undertaken in the USA, and one each in Canada, Korea, China, and The Netherlands. Four studies examined the effect of public release of performance data on consumer healthcare choices, and four on improving quality.There was low-certainty evidence that public release of performance data may make little or no difference to long-term healthcare utilisation by healthcare consumers (3 studies; 18,294 insurance plan beneficiaries), or providers (4 studies; 3,000,000 births, and 67 healthcare providers), or to provider performance (1 study; 82 providers). However, there was also low-certainty evidence to suggest that public release of performance data may slightly improve some patient outcomes (5 studies, 315,092 hospitalisations, and 7502 providers). There was low-certainty evidence from a single study to suggest that public release of performance data may have differential effects on disadvantaged populations. There was no evidence about effects on healthcare utilisation decisions by purchasers, or adverse effects.
Authors' conclusions: The existing evidence base is inadequate to directly inform policy and practice. Further studies should consider whether public release of performance data can improve patient outcomes, as well as healthcare processes.
Conflict of interest statement
DM: None known
NK: None known
ARD: None known
OO: None known
SF: None known
SM: None known
DP: None known
Figures



Update of
-
Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations.Cochrane Database Syst Rev. 2011 Nov 9;(11):CD004538. doi: 10.1002/14651858.CD004538.pub2. Cochrane Database Syst Rev. 2011. Update in: Cochrane Database Syst Rev. 2018 Sep 06;9:CD004538. doi: 10.1002/14651858.CD004538.pub3. PMID: 22071813 Free PMC article. Updated. Review.
Similar articles
-
Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations.Cochrane Database Syst Rev. 2011 Nov 9;(11):CD004538. doi: 10.1002/14651858.CD004538.pub2. Cochrane Database Syst Rev. 2011. Update in: Cochrane Database Syst Rev. 2018 Sep 06;9:CD004538. doi: 10.1002/14651858.CD004538.pub3. PMID: 22071813 Free PMC article. Updated. Review.
-
External inspection of compliance with standards for improved healthcare outcomes.Cochrane Database Syst Rev. 2016 Dec 2;12(12):CD008992. doi: 10.1002/14651858.CD008992.pub3. Cochrane Database Syst Rev. 2016. PMID: 27911487 Free PMC article. Review.
-
Printed educational materials: effects on professional practice and healthcare outcomes.Cochrane Database Syst Rev. 2020 Jul 31;8(8):CD004398. doi: 10.1002/14651858.CD004398.pub4. Cochrane Database Syst Rev. 2020. PMID: 32748975 Free PMC article.
-
Pharmaceutical policies: effects of financial incentives for prescribers.Cochrane Database Syst Rev. 2015 Aug 4;2015(8):CD006731. doi: 10.1002/14651858.CD006731.pub2. Cochrane Database Syst Rev. 2015. PMID: 26239041 Free PMC article. Review.
-
Effectiveness of external inspection of compliance with standards in improving healthcare organisation behaviour, healthcare professional behaviour or patient outcomes.Cochrane Database Syst Rev. 2011 Nov 9;(11):CD008992. doi: 10.1002/14651858.CD008992.pub2. Cochrane Database Syst Rev. 2011. Update in: Cochrane Database Syst Rev. 2016 Dec 02;12:CD008992. doi: 10.1002/14651858.CD008992.pub3. PMID: 22071861 Free PMC article. Updated. Review.
Cited by
-
Quality Assurance Measures and Mortality After Stroke.Dtsch Arztebl Int. 2021 Dec 17;118(50):857-863. doi: 10.3238/arztebl.m2021.0339. Dtsch Arztebl Int. 2021. PMID: 34730084 Free PMC article.
-
Referring physicians' intention to use hospital report cards for hospital referral purposes in the presence or absence of patient-reported outcomes: a randomized trial.Eur J Health Econ. 2024 Mar;25(2):293-305. doi: 10.1007/s10198-023-01587-6. Epub 2023 Apr 13. Eur J Health Econ. 2024. PMID: 37052802 Free PMC article. Review.
-
Mechanisms and impact of public reporting on physicians and hospitals' performance: A systematic review (2000-2020).PLoS One. 2021 Feb 24;16(2):e0247297. doi: 10.1371/journal.pone.0247297. eCollection 2021. PLoS One. 2021. PMID: 33626055 Free PMC article.
-
Managing the cancer backlog: a national population-based study of patient mobility, waiting times and 'spare capacity' for cancer surgery.Lancet Reg Health Eur. 2023 May 3;30:100642. doi: 10.1016/j.lanepe.2023.100642. eCollection 2023 Jul. Lancet Reg Health Eur. 2023. PMID: 37465324 Free PMC article.
-
Quality indicators in surgical oncology: systematic review of measures used to compare quality across hospitals.BJS Open. 2024 Mar 1;8(2):zrae009. doi: 10.1093/bjsopen/zrae009. BJS Open. 2024. PMID: 38513280 Free PMC article.
References
References to studies included in this review
DeVore 2016 {published data only}
-
- DeVore AD, Hammill BG, Hardy NC, Eapen ZJ, Peterson ED, Hernandez AF. Has public reporting of hospital readmission rates affected patient outcomes? Analysis of Medicare claims data. Journal of the American College of Cardiology 2016;67(8):963‐72. [PUBMED: 26916487] - PubMed
Farley 2002a {published data only}
-
- Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAPHS Performance Information on health plan choices by Iowa Medicaid. Medical Care Research and Review 2002;59(3):319‐36. [PUBMED: 12205831] - PubMed
Farley 2002b {published data only}
Flett 2015 {published data only}
-
- Flett KB, Ozonoff A, Graham DA, Sandora TJ, Priebe GP. Impact of mandatory public reporting of central line‐associated bloodstream infections on blood culture and antibiotic utilization in pediatric and neonatal intensive care units. Infection Control and Hospital Epidemiology 2015;36(8):878‐85. [PUBMED: 25913602] - PubMed
Ikkersheim 2013 {published data only}
Jang 2011 {published data only}
-
- Jang WM, Eun SJ, Lee C, Kim Y. Effect of repeated public releases on cesarean section rates. Journal of Preventative Medicine and Public Health 2011;44(1):2‐8. [PUBMED: 21483217] - PubMed
Joynt 2016 {published data only}
Liu 2017 {published data only}
Rinke 2015 {published data only}
Romano 2004 {published data only}
-
- Romano PS, Zhou H. Do well‐publicized risk‐adjusted outcomes reports affect hospital volume?. Medical Care 2004;42(4):367‐77. [PUBMED: 15076814] - PubMed
Tu 2009 {published data only}
-
- Tu JV, Donovan LR, Douglas SL, Wang JT, Austin PC, Alter DA, et al. Effectiveness of public report cards for improving the quality of cardiac care. The EFFECT study: a randomized trial. JAMA 2009;302(21):2330‐7. [PUBMED: 19923205] - PubMed
Zhang 2016 {published data only}
-
- Liu C, Zhang X, Wan J. Public reporting influences antibiotic and injection prescription in primary care: a segmented regression analysis. Journal of Evaluation in Clinical Practice 2015;21(4):597‐603. [PUBMED: 25902726] - PubMed
References to studies excluded from this review
Cavender 2015 {published data only}
Moscucci 2005 {published data only}
-
- Moscucci M, Eagle KA, Share D, Smith D, Franco AC, O'Donnell M, et al. Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases. Journal of the American College of Cardiology 2005;45(11):1759‐65. [PUBMED: 15936602] - PubMed
Paris 2013 {published data only}
-
- Paris B, Arahood T, Asche C, Amundson G. Lessons from voluntary reporting of Illinois hospital employee seasonal influenza vaccination rates (2009‐2013). Value in Health 2013;16(3):A96. - PubMed
Park 2011 {published data only}
Saratzis 2017 {published data only}
Additional references
Aggarwal 2017
Bender 2001
-
- Bender R, Lange S. Adjusting for multiple testing – when and how?. Journal of Clinical Epidemiology 2001;54(4):343‐9. [PUBMED: 11297884] - PubMed
Berwick 1990
-
- Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data. JAMA 1990;263(2):247‐9. - PubMed
Berwick 2003
-
- Berwick D, Jamer B, Coye M. Connections between quality measurements and improvement. Medical Care 2003;41(1):130‐8. - PubMed
Brook 1994
-
- Brook RH. Health care reform is on the way: do we want to compete on quality?. Annals of Internal Medicine 1994;120(1):84‐6. - PubMed
Burns 2016
-
- Burns EM, Pettengell C, Athanasiou T, Darzi A. Understanding the strengths and weaknesses of public reporting of surgeon‐specific outcome data. Health Affairs 2016;35(3):415‐21. - PubMed
Campanella 2016
Canaway 2017
Canaway 2018
-
- Canaway R, Mismark M, Dunt D, Prang KH, Kelaher M. "What is meant by public?": stakeholder views on strengthening impacts of public reporting of hospital performance data. Social Science & Medicine 2018;202:143‐50. [PUBMED: 29524870] - PubMed
Chung 2014
Damman 2010
Damman 2011
-
- Damman OC, Hendrik M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP. Consumers interpretation and use of comparative information on the quality of health care: the effect of presentation approaches. Health Expectations 25 May 2011 [Epub ahead of print]. [DOI: 10.1111/j.1369-7625.2011.00671.x] - DOI - PMC - PubMed
EPOC 2013
-
- Effective Practice, Organisation of Care (EPOC). Analysis in EPOC reviews. EPOC resources for review authors. Available at: epoc.cochrane.org/epoc‐resources‐review‐authors (accessed 22 August 2018).
EPOC 2017
-
- Effective Practice, Organisation of Care (EPOC). Reporting the effects of an intervention in EPOC reviews. Available at: epoc.cochrane.org/epoc‐resources‐review‐authors (accessed 22 August 2018).
Faber 2009
-
- Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality of care information? A systematic review. Medical Care 2009;47(1):1‐8. - PubMed
Fung 2008
-
- Fung C, Yee‐Wei L, Soeren M, Damberg C, Shekelle P. Systematic review: the evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine 2008;148(2):111‐23. - PubMed
Greenhalgh 2018
Harris 2008
-
- Harris KM, Beeuwkes Buntin M, The RAND Cooperation. Research Synthesis Report. Choosing a healthcare provider: the role of quality information. Princeton: Robert Wood Johnson Foundation, 2008.
Hendriks 2009
Hibbard 1997
-
- Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan: do large employers use the data?. Health Affairs 1997;16(6):172‐80. - PubMed
Hibbard 2007
-
- Hibbard JH, Peters E, Dixon A, Tusler M. Consumers competencies and the use of comparative quality information: it isn't just about literacy. Medical Care Research and Review 2007;64(4):379‐94. - PubMed
Hibbard 2009
-
- Hibbard JH. Using systematic measurement to target consumer activation strategies. Medical Care Research and Review 2009;66(1):9S‐27S. - PubMed
Hibbard 2010
-
- Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Medical Care Research and Review 2010;67(3):275‐93. - PubMed
Higgins 2011
-
- Higgins JPT, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.
Huwaldt 2004 [Computer program]
-
- Huwaldt JA, Steinhorst S, Paul G. Plot Digitizer. Version accessed 27 October 2015. USA: Huwaldt JA; SourceForge, 2004.
Ivers 2012
Kiernan 2015
-
- Kiernan F, Rahman F. Measuring surgical performance: a risky game?. Surgeon 2015;13(4):213‐7. - PubMed
Kolstad 2009
-
- Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Medical Care Research and Review 2009;66(Suppl 1):28S‐52S. - PubMed
Liu 2015
-
- Liu C, Zhang X, Wan J. Public reporting influences antibiotic and injection prescription in primary care: a segmented regression analysis. Journal of Evaluation in Clinical Practice 2015;21(4):597‐603. [PUBMED: 25902726] - PubMed
Liu 2016
Loeb 2004
-
- Loeb JM. The current state of performance measurement in health care. International Journal for Quality in Health Care 2004;16(Suppl 1):i5‐9. - PubMed
Marshall 2000
-
- Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283(14):1866‐74. [PUBMED: 10770149] - PubMed
Moher 1999
-
- Moher D, Cook DJ, Eastwood S, Olkin I, Drummon R, Stroup DF, et al. Improving the quality of reports of meta‐analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta‐analyses. Lancet 1999;354(9193):1896‐900. - PubMed
Moscelli 2017
Mukamel 1998
-
- Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State Cardiac Surgery Mortality Reports. Medical Care 1998;36(7):945‐54. - PubMed
Rosenthal 1998
-
- Rosenthal GE, Hammar PJ, Way LE, Shipley SA, Doner D, Wojtala B, et al. Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience. Joint Commission Journal of Quality Improvement 1998;24(7):347‐60. - PubMed
Schroll 2011
Schut 2005
-
- Schut FT, van de Ven WP. Rationing and competition in the Dutch health‐care system. Health Economics 2005;14(Suppl 1):S59‐74. - PubMed
Schwartz 2005
-
- Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821. [DOI: 10.1136/bmj.38614.449016.DE] - DOI - PMC - PubMed
Shahian 2017
-
- Shahian DM, Jacobs JP, Badhwar V, D'Agostino RS, Bavaria JE, Prager RL. Risk aversion and public reporting. Part 1: observations from cardiac surgery and interventional cardiology.. Annals of Thoracic Surgery 2017;104(6):2093‐101. [PUBMED: 29100643] - PubMed
Shekelle 2008
-
- Shekelle PG, Lim Y‐W, Mattke S, Damberg C, Southern California Evidence‐based Practice Centre, RAND Corporation. Does public release of performance results improve quality of care? A systematic review. London: The Health Foundation, 2008.
Sherman 2013
-
- Sherman KL, Gordon EJ, Mahvi DM, Chung J, Bentrem DJ, Holl JL, et al. Surgeons' perceptions of public reporting of hospital and individual surgeon quality. Medical Care 2013;51(12):1069‐75. - PubMed
Sirio 1996
-
- Sirio CA, McGee JL. Public reporting of clinical outcomes – the data needs of health care stakeholders. American Journal of Medical Quality 1996;11(1):S78‐81. - PubMed
Smith 2009
-
- Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system improvement. Experiences, Challenges and Prospects. Cambridge: Cambridge University Press, 2009.
Tang 2016
The King's Fund 2010
-
- Dixon A, Roberson R, Appleby J, Burge P, Devlin N, Magee H. Patient Choice. How patients choose and how providers respond. London: The King's Fund, 2010.
Wadhera 2017
Wang 2014
Wasfy 2015
-
- Wasfy JH, Borden WB, Secemsky EA, McCabe JM, Yeh RW. Public reporting in cardiovascular medicine: accountability, unintended consequences, and promise for improvement.. Circulation 2015;131(17):1518‐27. [PUBMED: 25918041] - PubMed
Werner 2009
Yang 2014
Zhang 2018 [pers comm]
-
- Zhang X (School of Medicine and Health Management, Tongji Medical College, HuaZhong University of Science and Technology, Wuhan, Hubei Province, China). [personal communication]. David Metcalfe (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences [NDORMS], University of Oxford, UK) 15 March 2018.
References to other published versions of this review
Ketelaar 2011
Publication types
MeSH terms
Grants and funding
LinkOut - more resources
Full Text Sources
Other Literature Sources
Medical