Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2018 Sep 6;9(9):CD004538.
doi: 10.1002/14651858.CD004538.pub3.

Impact of public release of performance data on the behaviour of healthcare consumers and providers

Affiliations
Review

Impact of public release of performance data on the behaviour of healthcare consumers and providers

David Metcalfe et al. Cochrane Database Syst Rev. .

Abstract

Background: It is becoming increasingly common to publish information about the quality and performance of healthcare organisations and individual professionals. However, we do not know how this information is used, or the extent to which such reporting leads to quality improvement by changing the behaviour of healthcare consumers, providers, and purchasers.

Objectives: To estimate the effects of public release of performance data, from any source, on changing the healthcare utilisation behaviour of healthcare consumers, providers (professionals and organisations), and purchasers of care. In addition, we sought to estimate the effects on healthcare provider performance, patient outcomes, and staff morale.

Search methods: We searched CENTRAL, MEDLINE, Embase, and two trials registers on 26 June 2017. We checked reference lists of all included studies to identify additional studies.

Selection criteria: We searched for randomised or non-randomised trials, interrupted time series, and controlled before-after studies of the effects of publicly releasing data regarding any aspect of the performance of healthcare organisations or professionals. Each study had to report at least one main outcome related to selecting or changing care.

Data collection and analysis: Two review authors independently screened studies for eligibility and extracted data. For each study, we extracted data about the target groups (healthcare consumers, healthcare providers, and healthcare purchasers), performance data, main outcomes (choice of healthcare provider, and improvement by means of changes in care), and other outcomes (awareness, attitude, knowledge of performance data, and costs). Given the substantial degree of clinical and methodological heterogeneity between the studies, we presented the findings for each policy in a structured format, but did not undertake a meta-analysis.

Main results: We included 12 studies that analysed data from more than 7570 providers (e.g. professionals and organisations), and a further 3,333,386 clinical encounters (e.g. patient referrals, prescriptions). We included four cluster-randomised trials, one cluster-non-randomised trial, six interrupted time series studies, and one controlled before-after study. Eight studies were undertaken in the USA, and one each in Canada, Korea, China, and The Netherlands. Four studies examined the effect of public release of performance data on consumer healthcare choices, and four on improving quality.There was low-certainty evidence that public release of performance data may make little or no difference to long-term healthcare utilisation by healthcare consumers (3 studies; 18,294 insurance plan beneficiaries), or providers (4 studies; 3,000,000 births, and 67 healthcare providers), or to provider performance (1 study; 82 providers). However, there was also low-certainty evidence to suggest that public release of performance data may slightly improve some patient outcomes (5 studies, 315,092 hospitalisations, and 7502 providers). There was low-certainty evidence from a single study to suggest that public release of performance data may have differential effects on disadvantaged populations. There was no evidence about effects on healthcare utilisation decisions by purchasers, or adverse effects.

Authors' conclusions: The existing evidence base is inadequate to directly inform policy and practice. Further studies should consider whether public release of performance data can improve patient outcomes, as well as healthcare processes.

PubMed Disclaimer

Conflict of interest statement

DM: None known

NK: None known

ARD: None known

OO: None known

SF: None known

SM: None known

DP: None known

Figures

1
1
Flowchart for study selection
2
2
Risk of bias graph: review authors' judgements about each risk of bias item, presented as percentages across all included studies. The blank spaces represent risk of bias criteria that were not applicable to the study design.
3
3
Risk of bias summary: review authors' judgements about each risk of bias item for each included study. The blank cells represent risk of bias criteria that were not applicable to the study design.

Update of

Similar articles

Cited by

References

References to studies included in this review

DeVore 2016 {published data only}
    1. DeVore AD, Hammill BG, Hardy NC, Eapen ZJ, Peterson ED, Hernandez AF. Has public reporting of hospital readmission rates affected patient outcomes? Analysis of Medicare claims data. Journal of the American College of Cardiology 2016;67(8):963‐72. [PUBMED: 26916487] - PubMed
Farley 2002a {published data only}
    1. Farley DO, Elliott MN, Short PF, Damiano P, Kanouse DE, Hays RD. Effect of CAPHS Performance Information on health plan choices by Iowa Medicaid. Medical Care Research and Review 2002;59(3):319‐36. [PUBMED: 12205831] - PubMed
Farley 2002b {published data only}
    1. Farley DO, Short PF, Elliott MN, Kanouse DE, Brown JA, Hays RD. Effect of CAPHS health plan performance information on plan choices by New Jersey. Health Services Research 2002;37(4):985‐1007. [PUBMED: 12205831] - PMC - PubMed
Flett 2015 {published data only}
    1. Flett KB, Ozonoff A, Graham DA, Sandora TJ, Priebe GP. Impact of mandatory public reporting of central line‐associated bloodstream infections on blood culture and antibiotic utilization in pediatric and neonatal intensive care units. Infection Control and Hospital Epidemiology 2015;36(8):878‐85. [PUBMED: 25913602] - PubMed
Ikkersheim 2013 {published data only}
    1. Ikkersheim D, Koolman X. The use of quality information by general practitioners: does it alter choices? A randomized clustered study. BMC Family Practice 2013;14:95. [PUBMED: 23824745] - PMC - PubMed
Jang 2011 {published data only}
    1. Jang WM, Eun SJ, Lee C, Kim Y. Effect of repeated public releases on cesarean section rates. Journal of Preventative Medicine and Public Health 2011;44(1):2‐8. [PUBMED: 21483217] - PubMed
Joynt 2016 {published data only}
    1. Joynt KE, Orav EJ, Zheng J, Jha AK. Public reporting of mortality rates for hospitalized Medicare patients and trends in mortality for reported conditions. Annals of Internal Medicine 2016;165(3):153‐60. [PUBMED: 27239794] - PMC - PubMed
Liu 2017 {published data only}
    1. Liu H, Herzig CTA, Dick AW, Furuya EY, Larson E, Reagan J, et al. Impact of state reporting laws on central line‐associated bloodstream infection rates in U.S. adult intensive care units. Health Services Research 2017;52(3):1079‐98. [PUBMED: 27451968] - PMC - PubMed
Rinke 2015 {published data only}
    1. Rinke ML, Bundy DG, Abdullah F, Colantuoni E, Zhang Y, Miller MR. State‐mandated hospital infection reporting is not associated with decreased pediatric health care‐associated infections. Journal of Patient Safety 2015;11:123‐34. [PUBMED: 24681422] - PMC - PubMed
Romano 2004 {published data only}
    1. Romano PS, Zhou H. Do well‐publicized risk‐adjusted outcomes reports affect hospital volume?. Medical Care 2004;42(4):367‐77. [PUBMED: 15076814] - PubMed
Tu 2009 {published data only}
    1. Tu JV, Donovan LR, Douglas SL, Wang JT, Austin PC, Alter DA, et al. Effectiveness of public report cards for improving the quality of cardiac care. The EFFECT study: a randomized trial. JAMA 2009;302(21):2330‐7. [PUBMED: 19923205] - PubMed
Zhang 2016 {published data only}
    1. Du X, Wang D, Wang X, Yang S, Zhang X. Exploring the transparency mechanism and evaluating the effect of public reporting on prescription: a protocol for a cluster randomized controlled trial. BMC Public Health 2015;21(15):277. [PUBMED: 25881035] - PMC - PubMed
    1. Liu C, Zhang X, Wan J. Public reporting influences antibiotic and injection prescription in primary care: a segmented regression analysis. Journal of Evaluation in Clinical Practice 2015;21(4):597‐603. [PUBMED: 25902726] - PubMed
    1. Liu C, Zhang X, Wang X, Wan J, Zhong F. Does public reporting influence antibiotic and injection prescribing to all patients? A cluster‐randomized matched‐pair trial in China. Medicine (Baltimore) 2016;95(26):e3965. [PUBMED: 27367995] - PMC - PubMed
    1. Tang Y, Liu C, Zhang X. Public reporting as a prescriptions quality improvement measure in primary care settings in China: variations in effects associated with diagnoses. Scientific Reports 2016;6:39361. [PUBMED: 27996026] - PMC - PubMed
    1. Wang X, Tang Y, Zhang X, Yin X, Du X, Zhang X. Effect of publicly reporting performance data of medicine use on injection use: a quasi‐experimental study. PLOS One 2014;9(10):e109594. [PUBMED: 25313853] - PMC - PubMed

References to studies excluded from this review

Cavender 2015 {published data only}
    1. Cavender MA, Joynt KE, Parzynski CS, Resnic FS, Rumsfeld JS, Moscucci M, et al. State mandated public reporting and outcomes of percutaneous coronary intervention in the United States. American Journal of Cardiology 2015;115(11):1494‐501. [PUBMED: 25891991] - PMC - PubMed
Moscucci 2005 {published data only}
    1. Moscucci M, Eagle KA, Share D, Smith D, Franco AC, O'Donnell M, et al. Public reporting and case selection for percutaneous coronary interventions: an analysis from two large multicenter percutaneous coronary intervention databases. Journal of the American College of Cardiology 2005;45(11):1759‐65. [PUBMED: 15936602] - PubMed
Paris 2013 {published data only}
    1. Paris B, Arahood T, Asche C, Amundson G. Lessons from voluntary reporting of Illinois hospital employee seasonal influenza vaccination rates (2009‐2013). Value in Health 2013;16(3):A96. - PubMed
Park 2011 {published data only}
    1. Park J, Konetzka RT, Werner RM. Performing well on nursing home report cards: does it pay off?. Health Services Research 2011;46(2):531‐54. [PUBMED: 21029093] - PMC - PubMed
Saratzis 2017 {published data only}
    1. Saratzis A, Thatcher A, Bath MF, Sidloff DA, Bown MJ, Shakespeare J, et al. Reporting individual surgeon outcomes does not lead to risk aversion in abdominal aortic aneurysm surgery. Annals of the Royal College of Surgeons of England 2017;99(2):161‐5. [PUBMED: 28071950] - PMC - PubMed

Additional references

Aggarwal 2017
    1. Aggarwal A, Lewis D, Mason M, Sullivan R, Meulen J. Patient mobility for elective secondary health care services in response to patient choice policies: a systematic review. Medical Care Research and Review: MCRR 2017;74(4):379‐403. [PUBMED: 27357394] - PMC - PubMed
Bender 2001
    1. Bender R, Lange S. Adjusting for multiple testing – when and how?. Journal of Clinical Epidemiology 2001;54(4):343‐9. [PUBMED: 11297884] - PubMed
Berwick 1990
    1. Berwick DM, Wald DL. Hospital leaders' opinions of the HCFA mortality data. JAMA 1990;263(2):247‐9. - PubMed
Berwick 2003
    1. Berwick D, Jamer B, Coye M. Connections between quality measurements and improvement. Medical Care 2003;41(1):130‐8. - PubMed
Brook 1994
    1. Brook RH. Health care reform is on the way: do we want to compete on quality?. Annals of Internal Medicine 1994;120(1):84‐6. - PubMed
Burns 2016
    1. Burns EM, Pettengell C, Athanasiou T, Darzi A. Understanding the strengths and weaknesses of public reporting of surgeon‐specific outcome data. Health Affairs 2016;35(3):415‐21. - PubMed
Campanella 2016
    1. Campanella P, Vukovic V, Parente P, Sulejmani A, Ricciardi W, Specchia ML. The impact of public reporting on clinical outcomes: a systematic review and meta‐analysis. BMC Health Services Research 2016;16:296. - PMC - PubMed
Canaway 2017
    1. Canaway R, Bismark M, Dunt D, Kelaher M. Perceived barriers to effective implementation of public reporting of hospital performance data in Australia: a qualitative study. BMC Health Services Research 2017;17(1):391. [PUBMED: 28592277] - PMC - PubMed
Canaway 2018
    1. Canaway R, Mismark M, Dunt D, Prang KH, Kelaher M. "What is meant by public?": stakeholder views on strengthening impacts of public reporting of hospital performance data. Social Science & Medicine 2018;202:143‐50. [PUBMED: 29524870] - PubMed
Chung 2014
    1. Chung SH, Seol HJ, Choi YS, Oh SY, Kim A, Bae CW. Changes in the cesarean section rate in Korea (1982‐2012) and a review of the associated factors. Journal of Korean Medical Science 2014;29(10):1341‐52. [PUBMED: 25368486 ] - PMC - PubMed
Damman 2010
    1. Damman OC, Hengel YK, Loon AJ, Rademakers J. An international comparison of Web‐based reporting about healthcare quality: content analysis. Journal of Medical Internet Research 2010;13(12):e8. - PMC - PubMed
Damman 2011
    1. Damman OC, Hendrik M, Rademakers J, Spreeuwenberg P, Delnoij DM, Groenewegen PP. Consumers interpretation and use of comparative information on the quality of health care: the effect of presentation approaches. Health Expectations 25 May 2011 [Epub ahead of print]. [DOI: 10.1111/j.1369-7625.2011.00671.x] - DOI - PMC - PubMed
EPOC 2013
    1. Effective Practice, Organisation of Care (EPOC). Analysis in EPOC reviews. EPOC resources for review authors. Available at: epoc.cochrane.org/epoc‐resources‐review‐authors (accessed 22 August 2018).
EPOC 2017
    1. Effective Practice, Organisation of Care (EPOC). Reporting the effects of an intervention in EPOC reviews. Available at: epoc.cochrane.org/epoc‐resources‐review‐authors (accessed 22 August 2018).
Faber 2009
    1. Faber M, Bosch M, Wollersheim H, Leatherman S, Grol R. Public reporting in health care: how do consumers use quality of care information? A systematic review. Medical Care 2009;47(1):1‐8. - PubMed
Fung 2008
    1. Fung C, Yee‐Wei L, Soeren M, Damberg C, Shekelle P. Systematic review: the evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine 2008;148(2):111‐23. - PubMed
Greenhalgh 2018
    1. Greenhalgh J, Dalkin S, Gibbons E, Wright J, Valderas JM, Meads D, et al. How do aggregated patient‐reported outcome measures data stimulate health care improvement? A realist synthesis. Journal of Health Services Research & Policy 20183;23(1):57‐65. [PUBMED: 29260592] - PMC - PubMed
Harris 2008
    1. Harris KM, Beeuwkes Buntin M, The RAND Cooperation. Research Synthesis Report. Choosing a healthcare provider: the role of quality information. Princeton: Robert Wood Johnson Foundation, 2008.
Hendriks 2009
    1. Hendriks M, Spreeuwenberg P, Rademakers J, Delnoij DM. Dutch healthcare reform: did it result in performance improvement of health plans? A comparison of consumer experiences over time. BMC Health Services Research 2009;9:167. [DOI: 10.1186/1472-6963-9-167] - DOI - PMC - PubMed
Hibbard 1997
    1. Hibbard JH, Jewett JJ, Legnini MW, Tusler M. Choosing a health plan: do large employers use the data?. Health Affairs 1997;16(6):172‐80. - PubMed
Hibbard 2007
    1. Hibbard JH, Peters E, Dixon A, Tusler M. Consumers competencies and the use of comparative quality information: it isn't just about literacy. Medical Care Research and Review 2007;64(4):379‐94. - PubMed
Hibbard 2009
    1. Hibbard JH. Using systematic measurement to target consumer activation strategies. Medical Care Research and Review 2009;66(1):9S‐27S. - PubMed
Hibbard 2010
    1. Hibbard JH, Greene J, Daniel D. What is quality anyway? Performance reports that clearly communicate to consumers the meaning of quality of care. Medical Care Research and Review 2010;67(3):275‐93. - PubMed
Higgins 2011
    1. Higgins JPT, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.
Huwaldt 2004 [Computer program]
    1. Huwaldt JA, Steinhorst S, Paul G. Plot Digitizer. Version accessed 27 October 2015. USA: Huwaldt JA; SourceForge, 2004.
Ivers 2012
    1. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard‐Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2012, Issue 6. [DOI: 10.1002/14651858.CD000259.pub3; PUBMED: 22696318] - DOI - PMC - PubMed
Kiernan 2015
    1. Kiernan F, Rahman F. Measuring surgical performance: a risky game?. Surgeon 2015;13(4):213‐7. - PubMed
Kolstad 2009
    1. Kolstad JT, Chernew ME. Quality and consumer decision making in the market for health insurance and health care services. Medical Care Research and Review 2009;66(Suppl 1):28S‐52S. - PubMed
Liu 2015
    1. Liu C, Zhang X, Wan J. Public reporting influences antibiotic and injection prescription in primary care: a segmented regression analysis. Journal of Evaluation in Clinical Practice 2015;21(4):597‐603. [PUBMED: 25902726] - PubMed
Liu 2016
    1. Liu C, Zhang X, Wang X, Zhang X, Wan J, Zhong F. Does public reporting influence antibiotic and injection prescribing to all patients? A cluster‐randomized matched‐pair trial in China. Medicine (Baltimore) 2016;95(26):e3965. [PUBMED: 27367995] - PMC - PubMed
Loeb 2004
    1. Loeb JM. The current state of performance measurement in health care. International Journal for Quality in Health Care 2004;16(Suppl 1):i5‐9. - PubMed
Marshall 2000
    1. Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA 2000;283(14):1866‐74. [PUBMED: 10770149] - PubMed
Moher 1999
    1. Moher D, Cook DJ, Eastwood S, Olkin I, Drummon R, Stroup DF, et al. Improving the quality of reports of meta‐analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta‐analyses. Lancet 1999;354(9193):1896‐900. - PubMed
Moscelli 2017
    1. Moscelli G, Siciliani L, Gutacker N, Cookson R. Socioeconomic inequality of access to healthcare: Does choice explain the gradient?. Journal of Health Economics 23 June 2017 [Epub ahead of print]. [DOI: 10.1016/j.jhealeco.2017.06.005; PUBMED: 28935158] - DOI - PubMed
Mukamel 1998
    1. Mukamel DB, Mushlin AI. Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State Cardiac Surgery Mortality Reports. Medical Care 1998;36(7):945‐54. - PubMed
Rosenthal 1998
    1. Rosenthal GE, Hammar PJ, Way LE, Shipley SA, Doner D, Wojtala B, et al. Using hospital performance data in quality improvement: the Cleveland Health Quality Choice experience. Joint Commission Journal of Quality Improvement 1998;24(7):347‐60. - PubMed
Schroll 2011
    1. Schroll JP, Moustgaard R, Gøtzsche PC. Dealing with substantial heterogeneity in Cochrane reviews. Cross‐sectional study. BMC Medical Research Methodology 2011;11:22. [PUBMED: 21349195] - PMC - PubMed
Schut 2005
    1. Schut FT, van de Ven WP. Rationing and competition in the Dutch health‐care system. Health Economics 2005;14(Suppl 1):S59‐74. - PubMed
Schwartz 2005
    1. Schwartz LM, Woloshin S, Birkmeyer JD. How do elderly patients decide where to go for major surgery? Telephone interview survey. BMJ 2005;331(7520):821. [DOI: 10.1136/bmj.38614.449016.DE] - DOI - PMC - PubMed
Shahian 2017
    1. Shahian DM, Jacobs JP, Badhwar V, D'Agostino RS, Bavaria JE, Prager RL. Risk aversion and public reporting. Part 1: observations from cardiac surgery and interventional cardiology.. Annals of Thoracic Surgery 2017;104(6):2093‐101. [PUBMED: 29100643] - PubMed
Shekelle 2008
    1. Shekelle PG, Lim Y‐W, Mattke S, Damberg C, Southern California Evidence‐based Practice Centre, RAND Corporation. Does public release of performance results improve quality of care? A systematic review. London: The Health Foundation, 2008.
Sherman 2013
    1. Sherman KL, Gordon EJ, Mahvi DM, Chung J, Bentrem DJ, Holl JL, et al. Surgeons' perceptions of public reporting of hospital and individual surgeon quality. Medical Care 2013;51(12):1069‐75. - PubMed
Sirio 1996
    1. Sirio CA, McGee JL. Public reporting of clinical outcomes – the data needs of health care stakeholders. American Journal of Medical Quality 1996;11(1):S78‐81. - PubMed
Smith 2009
    1. Smith PC, Mossialos E, Papanicolas I, Leatherman S. Performance measurement for health system improvement. Experiences, Challenges and Prospects. Cambridge: Cambridge University Press, 2009.
Tang 2016
    1. Tang Y, Liu C, Zhang X. Public reporting as a prescriptions quality improvement measure in primary care settings in China: variations in effects associated with diagnoses. Scientific Reports 2016;6:39361. [PUBMED: 27996026] - PMC - PubMed
The King's Fund 2010
    1. Dixon A, Roberson R, Appleby J, Burge P, Devlin N, Magee H. Patient Choice. How patients choose and how providers respond. London: The King's Fund, 2010.
Wadhera 2017
    1. Wahera RK, Anderson JD, Yeh RW. High‐risk percutaneous coronary intervention in public reporting states: the evidence, exclusion of critically ill patients, and implications. Current Heart Failure Reports 2017;14(6):514‐8. [PUBMED: 29101664] - PMC - PubMed
Wang 2014
    1. Wang X, Tang Y, Zhang X, Yin X, Du X, Zhang X. Effect of publicly reporting performance data of medicine use on injection use: a quasi‐experimental study. PLoS One 2014;9(10):e109594. [PUBMED: 25313853] - PMC - PubMed
Wasfy 2015
    1. Wasfy JH, Borden WB, Secemsky EA, McCabe JM, Yeh RW. Public reporting in cardiovascular medicine: accountability, unintended consequences, and promise for improvement.. Circulation 2015;131(17):1518‐27. [PUBMED: 25918041] - PubMed
Werner 2009
    1. Werner RM, Konetzka RM, Stuart EA, Norton EC, Polsky D, Park J. Impact of public reporting on quality of postacute care. Health Services Research 2009;44(4):1169‐87. - PMC - PubMed
Yang 2014
    1. Yang L, Liu C, Wang L, Yin X, Zhang X. Public reporting improves antibiotic prescribing for upper respiratory tract infections in primary care: a matched‐pair cluster‐randomized trial in China. Health Research Policy and Systems 2014;12:61. [PUBMED: 25304996] - PMC - PubMed
Zhang 2018 [pers comm]
    1. Zhang X (School of Medicine and Health Management, Tongji Medical College, HuaZhong University of Science and Technology, Wuhan, Hubei Province, China). [personal communication]. David Metcalfe (Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences [NDORMS], University of Oxford, UK) 15 March 2018.

References to other published versions of this review

Ketelaar 2011
    1. Ketelaar NA, Faber MJ, Flottorp S, Rygh LH, Deane KH, Eccles MP. Public release of performance data in changing the behaviour of healthcare consumers, professionals or organisations. Cochrane Database of Systematic Reviews 2011, Issue 11. [DOI: 10.1002/14651858.CD004538] - DOI - PMC - PubMed

LinkOut - more resources