Email updates

Keep up to date with the latest news and content from Systematic Reviews and BioMed Central.

Open Access Highly Accessed Research

The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis

Teryl K Nuckols12*, Crystal Smith-Spangler34, Sally C Morton5, Steven M Asch234, Vaspaan M Patel67, Laura J Anderson7, Emily L Deichsel7 and Paul G Shekelle128

Author Affiliations

1 Division of General Internal Medicine and Health Services Research, David Geffen School of Medicine at the University of California, 911 Broxton Ave, Los Angeles, CA 90024, USA

2 RAND Corporation, 1776 Main Street, Santa Monica, CA 90407, USA

3 VA Palo Alto Health Care System, 795 Willow Road, Menlo Park, CA 94025, USA

4 Stanford University, Palo Alto, CA 94305, USA

5 Department of Biostatistics, University of Pittsburgh, Graduate School of Public Health, Pittsburgh, PA 15261, USA

6 NCQA, 1100 13th street NW, Washington, DC 20005, USA

7 UCLA Jonathan and Karin Fielding School of Public Health, Los Angeles, CA 90024, USA

8 VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA

For all author emails, please log on.

Systematic Reviews 2014, 3:56  doi:10.1186/2046-4053-3-56

The electronic version of this article is the complete one and can be found online at: http://www.systematicreviewsjournal.com/content/3/1/56


Received:19 December 2013
Accepted:29 April 2014
Published:4 June 2014

© 2014 Nuckols et al.; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Abstract

Background

The Health Information Technology for Economic and Clinical Health (HITECH) Act subsidizes implementation by hospitals of electronic health records with computerized provider order entry (CPOE), which may reduce patient injuries caused by medication errors (preventable adverse drug events, pADEs). Effects on pADEs have not been rigorously quantified, and effects on medication errors have been variable. The objectives of this analysis were to assess the effectiveness of CPOE at reducing pADEs in hospital-related settings, and examine reasons for heterogeneous effects on medication errors.

Methods

Articles were identified using MEDLINE, Cochrane Library, Econlit, web-based databases, and bibliographies of previous systematic reviews (September 2013). Eligible studies compared CPOE with paper-order entry in acute care hospitals, and examined diverse pADEs or medication errors. Studies on children or with limited event-detection methods were excluded. Two investigators extracted data on events and factors potentially associated with effectiveness. We used random effects models to pool data.

Results

Sixteen studies addressing medication errors met pooling criteria; six also addressed pADEs. Thirteen studies used pre-post designs. Compared with paper-order entry, CPOE was associated with half as many pADEs (pooled risk ratio (RR) = 0.47, 95% CI 0.31 to 0.71) and medication errors (RR = 0.46, 95% CI 0.35 to 0.60). Regarding reasons for heterogeneous effects on medication errors, five intervention factors and two contextual factors were sufficiently reported to support subgroup analyses or meta-regression. Differences between commercial versus homegrown systems, presence and sophistication of clinical decision support, hospital-wide versus limited implementation, and US versus non-US studies were not significant, nor was timing of publication. Higher baseline rates of medication errors predicted greater reductions (P < 0.001). Other context and implementation variables were seldom reported.

Conclusions

In hospital-related settings, implementing CPOE is associated with a greater than 50% decline in pADEs, although the studies used weak designs. Decreases in medication errors are similar and robust to variations in important aspects of intervention design and context. This suggests that CPOE implementation, as subsidized under the HITECH Act, may benefit public health. More detailed reporting of the context and process of implementation could shed light on factors associated with greater effectiveness.

Keywords:
Medical order entry systems; Drug toxicity/prevention and control; Hospitals; Adverse drug event; Medication error

Background

The Health Information Technology for Economic and Clinical Health (HITECH) Act of 2009 incentivizes the adoption of health information technology by US hospitals. This Act, part of the American Reinvestment and Recovery Act, allocates up to $29 billion over 10 years for the implementation and 'meaningful use' of electronic health records by hospitals and healthcare providers [1]. Hospitals that satisfy meaningful use criteria can receive millions of dollars. Implementing computerized provider order entry (CPOE) with clinical decision support systems (CDSS) that check for allergies and drug-drug interactions is one of several basic (Stage 1) criteria for meaningful use by hospitals [2]. As of 2008, approximately 9% of general acute care hospitals had at least basic electronic health record (EHR) systems including CPOE for medications. By 2012, 44% had such systems, specifically, 38% of small, 47% of medium, and 62% of large hospitals [3]. Thus, despite the financial incentives, about half of small and medium hospitals and almost 40% of large hospitals had not adopted CPOE with CDSS in the most recent survey.

The primary potential benefit of adopting CPOE is reducing patient injuries caused by medication errors, called preventable adverse drug events (pADEs) [4-6]. Counterbalancing this is concern about unintended adverse consequences [7-9], including increases in medication errors and even mortality, which have been detected in some hospitals after implementation of CPOE [10,11]. To date, no systematic review has examined net effects on pADEs, the primary outcome of interest for this intervention. Previous reviews have, instead, focused almost exclusively on an intermediate outcome, medication errors. However, not all medication errors pose an equal risk of causing injury. Errors in timing, for example, are generally less risky than giving a medication to the wrong patient. Many commonly used medications, such as anti-hypertensives and antibiotics, have sufficiently long half-lives that receiving a dose an hour or two late has little clinical effect. By contrast, receiving an anti-hypertensive or antibiotic intended for someone else poses risks of low blood pressure or an allergic reaction. In one study at six hospitals, only about 20% of medication errors led to pADEs [12]. Thus, the effect of CPOE on the patient outcome of pADEs is an important clinical and policy question that has remained unanswered, until now.

In addition to focusing on medication errors rather than pADEs, previous systematic reviews have reached conflicting conclusions about the effects of CPOE on medication errors in acute care settings. Some have concluded that CPOE reduces errors, whereas others argue that net effects remain uncertain [4,5,13-42]. This controversy stems, in part, from the fact that the association between CPOE implementation and medication errors has exhibited substantial heterogeneity across primary studies [37]. Three basic types of factors could explain such variability: intervention factors, such as differences in how the intervention is designed and implemented; contextual factors, such as differences in patient populations and settings; and methodological factors, such as differences in study design and execution [43].

Uncertainty about the effects of CPOE on patient outcomes and its variable effects on medication errors may contribute to the reluctance of some hospitals and physicians to adopt CPOE, despite the financial incentives available via HITECH. Consequently, our primary objective in this study was to quantitatively assess the effectiveness of CPOE at reducing pADEs in hospital-related acute care settings. Our secondary objective was to identify factors contributing to variability in effectiveness at reducing medication errors. This analysis is timely as several studies have been published recently and, therefore, were not included in previous reviews and meta-analyses [4,13,34,37,41], enabling us to examine effects on pADEs and reasons for heterogeneity.

Methods

We adhered to recommendations in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement [44,45], including developing the protocol before undertaking the analysis.

Data sources and searches

First, we developed search strategies for eight databases: MEDLINE; Cochrane Library; Econlit; Campbell Collaboration; the Agency for Healthcare Research and Quality (AHRQ) Health Information Technology Library, Health Information Technology Bibliography, Health Information Technology Costs and Benefits Database Project, and PSNET; Information Service Center for Reviews and Dissemination at the University of York; Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre), University of London; Oregon Health Sciences Searchable CPOE Bibliography; and Health Systems Evidence, McMaster University. A number of search terms, such as 'order entry' and 'electronic prescribing' (see Additional file 1), were chosen and strategies developed, in part based on a search strategy published by Eslami et al. [4].

Additional file 1. Appendix.

Format: DOC Size: 789KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

We used this strategy to search the eight databases for systematic reviews of CPOE or CDSS that might contain potentially relevant primary studies (last updated September 23, 2013) (Figure 1). Next, we used the same strategy to search the eight databases for potentially relevant primary studies that were published after two large previous systematic reviews on CPOE (January 1, 2007 to September 23, 2013) [4,13]. In addition, we hand-searched nine websites (AHRQ HIT Library, AHRQ PSNET, National Patient Safety Foundation, Joint Commission, Leapfrog Group, Micromedex, Institute for Healthcare Improvement), the Web of Science, and bibliographies of other publications known to us.

thumbnailFigure 1. Summary of evidence search and selection.

Study selection

We included peer-reviewed studies, regardless of language or design, if they compared CPOE with paper-order entry and examined either of our two primary outcomes, rates of pADEs or medication errors, across a variety of clinical conditions. Eligible settings included adult medical or surgical wards, adult medical or surgical intensive care units (ICUs), emergency departments, or the entire hospital. To reduce unwarranted variability due to contextual and methodological factors, we excluded studies that were from non-hospital settings; that addressed events limited to specific conditions (for example, infections) or types of errors (for example, allergy alerts); or that compared events in highly dissimilar patient care units. As minimum criteria for study quality, we excluded studies that did not describe methods for detecting medication events, or that used incident reporting alone, which detects 0.2–-6% of events [46]. We also excluded pediatric studies because including them would increase heterogeneity: children comprise only 6% of hospitalized patients whereas ADEs disproportionately affect older adults [12,47,48].

Two investigators independently screened the article titles and then abstracts for eligibility. We obtained full-text articles when either investigator found the abstract (or title, if the abstract was unavailable) potentially eligible. Disagreements about the eligibility of full-text articles were resolved by consensus, with a third investigator participating for ties.

Data extraction and quality assessment

We defined pADEs as injuries to patients due to medication errors. Medication errors were defined as errors in the process of prescribing, transcribing, dispensing, or administration of a medication, which had the potential to or actually did cause harm. To focus on errors involving relatively higher risk, we excluded, when reported, 'errors' described as having no or almost no potential for harm as well as incomplete or illegible orders, disallowed abbreviations, disallowed drug names, and medications given at the wrong time (see Additional file 1).

Two investigators independently extracted data from each study using a standardized form (see Additional file 1). Disagreements were resolved by consensus, with a third investigator adjudicating ties. Extracted elements included numbers of pADEs and medication errors meeting study definitions, units of exposure to risk of pADEs or medication errors (for example, number of orders, dispensed doses, admissions, or patient days). When studies reported rates or proportions rather than these elements, variance could not be estimated, so the studies could not be included in pooled effect calculations and thus we qualitatively summarized their results instead.

From the studies included in the pooled analysis of medication errors, we extracted several elements related to intervention design and implementation, context, and study methods. Elements related to intervention design included: CPOE developer (homegrown versus commercial); and presence or absence of CDSS, CDSS sophistication (basic, moderate, or advanced; see Table 1 for definitions). When information about the system developer and CDSS were missing from the published article, we contacted the original authors.

Table 1. Characteristics of included studies

Elements related to implementation were based on an AHRQ report addressing context-sensitive patient safety practices, including CPOE. These included: factors influencing the decision to adopt, factors facilitating implementation, and aspects of implementation described in the studies, as well as timing, extent of implementation (limited number of units versus hospital-wide), and whether use was mandatory (see Additional file 1 for details) [72].

Contextual elements included setting/population (type of clinical unit within the hospital, academic status, public versus private hospital, hospital size, country, primary language in country, payer mix), and baseline proportion of hospitalizations affected by medication errors.

Methodological elements included type of study design, event detection methods, items related to study quality (adapted from relevant reporting criteria in the Standards for Quality Improvement Reporting Excellence; SQUIRE) [73], and funding source.

Data synthesis and analysis

Using the DerSimonian–Laird random effects model [74], we conducted meta-analyses for two outcomes (pADEs and medication errors) for all eligible studies combined, and for different subgroups of studies as described below. For each eligible study and outcome measure, we calculated a risk ratio (RR) as the number of events per unit of exposure in the CPOE group divided by events per unit of exposure in the paper-order entry group. Units of exposure varied across studies. If a study provided more than one unit of exposure, we selected the unit most commonly used in the included studies.

Within each meta-analysis, we tested the heterogeneity of the log-transformed RRs using Q and I2 statistics [75]. Heterogeneity was present when the I2 statistic was 50% or more and the P-value for the Q statistic was 0.05 or less.

We conducted two sensitivity analyses, removing one study at a time from each meta-analysis to assess the influence of each individual study, and testing whether the choice of units of exposure affected results. To assess publication bias, we examined funnel plots, Begg and Mazumdar’s rank correlation test, and Egger’s regression intercept test [76].

Intervention design and implementation, contextual, and methodological factors

A priori, we identified nine factors that might be associated with heterogeneity in medication errors across studies. Intervention design factors included type of CPOE developer (homegrown versus commercial), presence or absence of CDSS, and sophistication of CDSS (basic, moderate, or advanced). Intervention implementation factors included scope of implementation (hospital-wide versus limited) and timing of CPOE implementation (year CPOE was implemented or, if missing, the year the study was published). Contextual factors included country (US versus non-US) and baseline proportion of hospitalizations affected by medication errors. Methodological design factors included study design (pre-post versus other designs) and event detection methods (pharmacist order review versus more comprehensive methods). For each discrete factor, we conducted a subgroup analysis when there were at least three studies per subgroup, for example, pre/post design versus other design. For each continuous factor, we conducted a meta-regression using the factor as the sole predictor. In each meta-regression, we pooled log-transformed RRs, and presented the pooled results on the original RR scale.

Pooled meta-analyses were conducted using Comprehensive Meta-analysis, V2 (Biostat, Englewood, NJ, USA); meta-regression analyses were conducted in STATA (V13) (StataCorp LP, College Station, TX, USA).

Results

We screened 4,891 potentially eligible records, including the bibliographies of 32 systematic reviews on CPOE or CDSS [4,5,13-42], and then examined 93 full-text articles on CPOE. Of these 93 full-text articles, 74 were excluded: 32 did not test the effectiveness of CPOE, 3 addressed non-hospital settings, 6 addressed pediatric settings, 5 used incident reporting alone to detect events, 1 did not describe event detection methods, 16 addressed outcomes other than medication errors or pADEs (for example, workflow or cost), 5 addressed errors limited to specific conditions, 1 addressed specific types of errors, 2 (1 in French) addressed errors that were excluded because they posed a lower risks of harm, 1 (in Spanish) compared event rates in dissimilar clinical units (obstetric and oncology), and 2 were duplicate publications of articles meeting the selection criteria (Figure 1; see Additional file 1, Additional file 2).

Additional file 2. PRISMA Checklist.

Format: DOC Size: 64KB Download file

This file can be viewed with: Microsoft Word ViewerOpen Data

The remaining 19 original articles met the selection criteria and addressed medication errors; 7 of these also addressed pADEs (Table 1) [11,49,50,53-55,57-68,70]. Of these 19 studies, 3 omitted the data needed to estimate variance and, therefore, were excluded from pooled effect calculations, resulting in 16 eligible studies, including 6 that addressed pADEs [57,62,67].

Of the 16 studies, half were based in the US, including two in community hospitals [11,55]. Thirteen studies used pre/post designs [11,49,50,53-55,58-60,63,65,68,70], two compared similar units within a hospital during the same time period [64,66], and one compared changes over time between intervention and control units (differences in differences design) [61]. Definitions of medication errors and the methods used to detect them varied across studies (see Additional file 1). Seven studies identified events using data from routine pharmacist review of medication orders [49,54,55,58,61,64,68]. One study provided information on reviewer training [11], three on blinding of reviewers [11,59,64], and none on reliability. The baseline percentage of hospitalizations affected by medication errors ranged from 3.6% [49] to 99.9% [60].

Nine studies assessed commercially developed CPOE systems [11,49,50,54,55,61,64,65,70], six evaluated homegrown systems [53,58,59,63,66,68], and one examined both [60]. No two studies assessed the same commercial system. CDSS was present in twelve studies [11,55,58-61,63-66,68,70], and absent in four [49,50,53,54]; we contacted and obtained responses from authors for three of the studies (Table 1).

For all but one study [58], most of the desired information on implementation was missing (see Additional file 1). Based on the information that was reported, ten studies described the use of CPOE as mandatory at one or more sites [49,50,53,55,58-61,63,64]. CPOE was implemented hospital-wide in four studies [11,58,65,70], in the emergency department in two studies [49,68], and in a limited number of inpatient units in the rest. Four studies were conducted in complex organizations with facilities in multiple communities [55,59,63,65], another study was in a large hospital with affiliated clinics [49], and another was in community hospitals [11]. Past experience with information technology was reported in seven studies [49,50,55,58,59,63,65]. Three studies reported that organizational leadership influenced the adoption decision [55,58,65], and four stated that staff training and education facilitated implementation [53,54,58,66]. One study mentioned the role of staff time to learn CPOE, a person to lead implementation, extensive project management, an implementation timeline, teamwork, and patient safety culture related to CPOE [58]. Another study described the effects of having a responsible person, local tailoring, and teamwork [65].

The three studies omitted from the pooled analysis due to lack of variance estimates were similar to the included studies. They were conducted in the US in medium to large hospitals, including one in a community hospital. One study evaluated a commercially developed system [67]; the other two did not report the developer. Two studies included CDSS [57,67]. All three used pre/post designs, one detected events using pharmacist review of medication orders [67], and none reported reviewer training, blinding, or reliability. These studies also did not report implementation context or processes in detail [62,67], except for one, which discussed financial considerations and leadership [57].

Primary outcome: preventable adverse drug events

Of the 19 studies, 7 assessed pADEs [11,59,60,62-64,70]. For the six studies in the pooled analysis, RRs ranged from 0.17 to 0.81. Overall, CPOE was associated with about half as many pADEs as paper-order entry (pooled RR = 0.47, 95% CI 0.31 to 0.71). Studies were heterogeneous (I2 = 69%) (Figure 2). Serial removal of each study did not substantially influence results (pooled RR range 0.40 to 0.58). There was no evidence of publication bias using a funnel plot, or Begg and Mazumdar’s test (see Additional file 1). For one study excluded from the pooled analysis due to lack of data on variance, we calculated an RR of 0.11 [62].

thumbnailFigure 2. Meta-analysis: relative risk of preventable adverse drug events using computerized provider order entry (CPOE) versus paper-order entry in hospital acute care settings. Units of exposure: *1,000 patient days; admissions.

Secondary outcome: medication errors

All 19 studies meeting selection criteria assessed medication errors [11,49,50,53-55,57-68,70]. Across the 16 studies eligible for the pooled analysis, RRs ranged from 0.16 to 2.08. The pooled estimate showed that medication errors were approximately half as common when providers used CPOE than when they used paper-order entry (pooled RR = 0.46, 95% CI 0.35 to 0.60). The studies were highly heterogeneous (I2 = 99%) (Figure 3). Results were robust to serial removal of each individual study (pooled RR range 0.42 to 0.49), and to selection of an alternative unit of exposure in the four studies where that was possible (pooled RR = 0.45, 95% CI 0.34 to 0.59). There was no evidence of publication bias using a funnel plot, or Begg and Mazumdar’s test (see Additional file 1).

thumbnailFigure 3. Meta-analysis: relative risk of medication errors using computerized provider order entry (CPOE) versus paper-order entry in hospital acute care settings. Units of exposure: *1,000 patient days; †orders; ‡dispensed doses; §admissions.

Two studies included in the pooled analysis reported increases in medication errors after the introduction of CPOE, however, both also reported statistically significant decreases in preventable adverse drug events [11,70]. A third study, excluded due to lack of data on variance, also showed an increase in errors and a decrease in pADEs, but statistical testing was not performed [62].

For two studies excluded from the pooled analysis due to lack of data on variance, we calculated RRs of 0.61 [67], and 1.73, respectively [62]. In the third such study, the authors reported a 50% decline in medication errors (see Additional file 1) [57].

Intervention design and implementation, contextual, and methodological factors

Six of the a priori subgroup analyses met the requirement to have at least three studies per subgroup and were, therefore, conducted (two were on one variable, CDSS) (Figure 3). Two univariate meta-regression analyses were able to examine whether baseline medication error rate or year of publication (a proxy for maturity of CPOE intervention; date of implementation was frequently missing) predicted effectiveness.

Of five intervention design and implementation factors examined, none reached the conventional level of statistical significance, including type of developer (commercial 0.56 (95% CI 0.36 to 0.85) versus homegrown 0.37 (0.29 to 0.47)), type of CDSS (present 0.44 (0.32 to 0.62) versus absent 0.51 (0.31 to 0.87), and basic 0.40 (0.38 to 0.87) versus moderate or advanced 0.51 (0.26 to 0.97)), and scope of implementation (hospital-wide 0.78 (0.36 to 1.70) versus limited 0.38 (0.32 to 0.46)). Year of publication was not associated with differential effectiveness.

Two contextual factors were evaluated. Studies performed in the US showed greater effectiveness than non-US studies, but this difference was not statistically significant. As the baseline percentage of hospitalizations associated with medication errors increased from 3.6% to 99.9% (data available for 12 studies), the predicted RR of medication errors with CPOE decreased from 1.90 to 0.08 (P < 0.001).

Regarding methodological factors, studies that used pharmacist order review reported greater effectiveness than studies using more comprehensive event detection methods, although this difference was not statistically significant. Almost all studies used pre/post designs so this subgroup analysis was not conducted.

Discussion

The principal finding of this analysis is that CPOE is associated with a significant reduction in pADEs (hat is, the patient injuries it was designed to prevent) in adult hospital-related acute care settings. Specifically, compared with using paper orders, using CPOE was associated with about half as many pADEs. Medication errors, likewise, were also about half as common with CPOE as with paper-order entry, and the reduction was generally similar across studies with different intervention designs and different implementation, contextual, and methodological characteristics. There were no statistically significant differences in effect between commercial and homegrown systems, with or without CDSS of differing sophistication levels, and between hospital-wide or more limited implementations. The baseline rate of hospitalizations associated with medication errors was significantly associated with effectiveness, as increasing baseline rates of errors were associated with increasing effectiveness. This is expected, because, with few errors, there can be little to change.

Our pooled analysis is conclusive that CPOE is associated with a reduction in pADEs. Shamliyan et al. examined ADEs that might or might not have been related to medication errors, and, therefore, were not as likely to be affected by CPOE. These authors observed significant declines in only three of seven studies (including pediatric ones), and did not perform a pooled analysis [37].

With regards to the overall pooled result for medication errors, our findings are generally consistent with those of earlier, more limited systematic reviews and meta-analyses [34,37,41]. Radley and colleagues also found that medication error rates declined by about half with CPOE implementation (48%, 95% CI 41 to 55%), using a small set of early studies [34]. Van Rosse and colleagues observed greater effectiveness with CPOE than we did (RR of medication errors = 0.08, 95% CI 0.01 41 to 0.76), but examined only three diverse studies [41]. Shamliyan and colleagues found that CPOE was slightly more effective than we did (odds ratio for medication errors = 0.34, 95% CI 0.22 41 to 0.52), based on inpatient and outpatient studies from before 2006 [37]. In comparison to these previous studies, we were able to identify a greater number of relevant articles despite having more restrictive selection criteria (see Additional file 1), enabling us to explore reasons for study heterogeneity.

Also like previous reviews [37], we observed substantial variability across studies in the effectiveness of CPOE at reducing medication errors. It has long been suspected that variability in the effectiveness of a complex sociotechnical intervention such as CPOE may be related not only to intervention design but also to context and implementation factors [16,77,78]. However, across the intervention design and implementation as well as contextual variables that we assessed, we did not see any statistically significant differences in the associations between CPOE use and reductions in medication errors. Two studies of commercial CPOE systems in hospital-wide implementations reported increases in medication errors but reductions in pADEs [11,70]. One potential explanation for these seemingly contradictory results is that the CPOE systems may have created new medication errors at lower risk for causing ADEs (such as concurrent submission of duplicate orders due to order sets) but reduced medication errors at higher risk of causing ADEs (such as serious drug-drug interactions). Alternatively, CPOE may have made errors easier to detect. The potential to create new types of low-risk medication errors calls attention to the importance of tailoring the CPOE system to the local environment because such errors place a time burden on providers.

This analysis has limitations. We relied on 32 previous systematic reviews to detect primary studies published before 2007. Because each review detected a slightly different set of publications (see Additional file 1), performing our own search of that period would have been unlikely to detect additional studies. We excluded pediatric studies instead of examining population age as a subgroup because these groups differ in their risk for experiencing medication errors and pADEs. Future investigators could evaluate the feasibility of conducting a similar meta-analysis for pediatric populations. We also excluded studies that relied upon incident reporting or did not describe event detection methods, considering these to be minimum criteria for study quality. The number of studies that examined pADEs was not large, but all studies detected declines. Most studies were conducted in academic centers, limiting generalizability to community hospitals. Finally, the included studies all used limited methods, including using pre/post designs and lacking robust data-collection methods.

Conclusion

Implementing CPOE is associated with a greater than 50% decline in pADE rates in hospital-related settings, although results vary. Medication errors decline to a similar degree. Changes in medication errors appear to be consistent across commercial and homegrown systems, with or without clinical decision support, and in individual units or hospital-wide implementations. Many context and implementation variables have, unfortunately, not been reported sufficiently to assess their association with effectiveness. Overall, these findings suggest that the CPOE requirements for meaningful use under the HITECH Act may benefit public health. Knowledge about how to make CPOE more effective would be greatly facilitated by greater reporting of context and implementation details.

Abbreviations

ADE: Adverse drug event; pADE: Preventable adverse drug event; CDSS: Clinical decision support systems; CPOE: Computerized provider order entry; ED: emergency department; EHR: Electronic health record; HITECH: Health Information Technology for Economic and Clinical Health; ICU: intensive care unit.

Competing interests

The authors have no conflicts of interest with the work.

Authors’ contributions

TKN, CSS, PGS: conception and design, data collection and analysis, manuscript writing; SCM data analysis and manuscript writing; SMA conception and design; VMP, LJA, and ELD: data collection and analysis. All authors read and approved the final manuscript.

Acknowledgements

This work was funded through a Mentored Clinical Scientist Career Development Award (K08) from the Agency for Healthcare Research and Quality (to TKN; grant number HS17954). The funder played no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. There were no other funding sources for this work. The assistance of Lance Tan in preparing the manuscript is greatly appreciated.

References

  1. Blumenthal D: Wiring the health system–origins and provisions of a new federal program.

    N Engl J Med 2011, 365(24):2323-2329. OpenURL

  2. Classen DC, Bates DW: Finding the meaning in meaningful use.

    N Engl J Med 2011, 365(9):855-8.

    doi:10.1056/NEJMsb1103659

    OpenURL

  3. Desroches CM, Charles D, Furukawa MF, Joshi MS, Kralovec P, Mostashari F, Worzala C, Jha AK: Adoption of electronic health records grows rapidly, but fewer than half of US hospitals had at least a basic system in 2012.

    Health Aff (Millwood) 2013, 32(8):1478-1485. OpenURL

  4. Eslami S, de Keizer NF, Abu-Hanna A: The impact of computerized physician medication order entry in hospitalized patients–a systematic review.

    Int J Med Inform 2008, 77(6):365-376. OpenURL

  5. Weir CR, Staggers N, Phansalkar S: The state of the evidence for computerized provider order entry: a systematic review and analysis of the quality of the literature.

    Int J Med Inform 2009, 78(6):365-374. OpenURL

  6. Bates D, Cullen D, Laird N, Petersen L, Small S, Servi D, Laffel G, Sweitzer B, Shea B, Hallisey R, Vandervliet M, Nemeskal R, Leape LL: Incidence of adverse drug events and potential adverse drug events: implications for prevention: ADE prevention study group.

    JAMA 1995, 274(1):29-34. OpenURL

  7. Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH: Types of unintended consequences related to computerized provider order entry.

    J Am Med Inform Assoc 2006, 13(5):547-556. OpenURL

  8. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, Strom BL: Role of computerized physician order entry systems in facilitating medication errors.

    JAMA 2005, 293(10):1197-1203. OpenURL

  9. Magrabi F, Ong MS, Runciman W, Coiera E: Using FDA reports to inform a classification for health information technology safety problems.

    J Am Med Inform Assoc 2012, 19(1):45-53. OpenURL

  10. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, Bayir H, Orr RA: Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system.

    Pediatrics 2005, 116(6):1506-1512. OpenURL

  11. Leung AA, Keohane C, Amato M, Simon SR, Coffey M, Kaufman N, Cadet B, Schiff G, Zimlichman E, Seger DL, Yoon C, Song P, Bates DW: Impact of vendor computerized physician order entry in community hospitals.

    J Gen Intern Med 2012, 7:801-7.

    doi:10.1007/s11606-012-1987-7. Epub 2012 Jan 21

    OpenURL

  12. Hug B, Witkowski D, Sox C, Keohane C, Seger D, Yoon C, Matheny M, Bates D: Adverse drug event rates in six community hospitals and the potential impact of computerized physician order entry for prevention.

    J Gen Intern Med 2010, 25(1):31-38. OpenURL

  13. Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U: The effect of electronic prescribing on medication errors and adverse drug events: a systematic review.

    J Am Med Inform Assoc 2008, 15(5):585-600. OpenURL

  14. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, Samsa G, Hasselblad V, Williams JW, Musty MD, Wing L, Kendrick AS, Sanders GD, Lobach D: Effect of clinical decision-support systems: a systematic review.

    Ann Intern Med 2012, 157(1):29-43. OpenURL

  15. Buntin MB, Burke MF, Hoaglin MC, Blumenthal D: The benefits of health information technology: a review of the recent literature shows predominantly positive results.

    Health Aff (Millwood) 2011, 30(3):464-471. OpenURL

  16. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG: Systematic review: impact of health information technology on quality, efficiency, and costs of medical care.

    Ann Intern Med 2006, 144(10):742-752. OpenURL

  17. Colpaert K, Decruyenaere J: Computerized physician order entry in critical care.

    Best Pract Res Clin Anaesthesiol 2009, 23(1):27-38. OpenURL

  18. Durieux P, Trinquart L, Colombet I, Nies J, Walton R, Rajeswaran A, Rege Walther M, Harvey E, Burnand B: Computerized advice on drug dosage to improve prescribing practice.

    Cochrane Database Syst Rev 2008, 3:CD002894.

    doi:10.1002/14651858.CD002894.pub2

    OpenURL

  19. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, Sam J, Haynes RB: Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review.

    JAMA 2005, 293(10):1223-1238. OpenURL

  20. Goldzweig CL, Towfigh A, Maglione M, Shekelle PG: Costs and benefits of health information technology: new trends from the literature.

    Health Aff 2009, 28(2):w282-w293. OpenURL

  21. Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB: Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review.

    Implement Sci 2011, 6(1):89. OpenURL

  22. Jamal A, McKenzie K, Clark M: The impact of health information technology on the quality of medical and health care: a systematic review.

    HIM J 2009, 38(3):26-37. OpenURL

  23. Jones SS, Rudin RS, Perry T, Shekelle PG: Health information technology: an updated systematic review with a focus on meaningful use.

    Ann Intern Med

    In Press

    OpenURL

  24. Kaushal R, Shojania KG, Bates DW: Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review.

    Arch Intern Med 2003, 163(12):1409-1416. OpenURL

  25. Kawamoto K, Lobach DF: Clinical decision support provided within physician order entry systems: a systematic review of features effective for changing clinician behavior.

    AMIA Annu Symp Proc 2003, 361-365. OpenURL

  26. Kuperman G, Bobb A, Payne T, Avery A, Gandhi T, Burns G, Classen D, Bates D: Medication-related clinical decision support in computerized provider order entry systems: a review.

    J Am Med Inform Assoc 2007, 14(1):29-40. OpenURL

  27. Manias E, Williams A, Liew D: Interventions to reduce medication errors in adult intensive care: a systematic review.

    Br J Clin Pharmacol 2012, 74(3):411-423. OpenURL

  28. Maslove D, Rizk N, Lowe H: Computerized physician order entry in the critical care environment: a review of current literature.

    J Am Med Inform Assoc 2007, 14(1):29-40.

    Epub 2006 Oct 26

    OpenURL

  29. McKibbon K, Lokker C, Handler S, Dolovich L, Holbrook A, O’Reilly D, Tamblyn R, Hemens B, Basu R, Troyan S, Roshanov P: The effectiveness of integrated health information technologies across the phases of medication management: a systematic review of randomized controlled trials.

    J Am Med Inform Assoc 2012, 19(1):22-30. OpenURL

  30. Mollon B, Chong JJ, Holbrook A, Sung M, Thabane L, Foster G: Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials.

    BMC Med Inform Decis Mak 2009, 9:11.

    doi:10.1186/1472-6947-9-11

    OpenURL

  31. Niès J, Colombet I, Degoulet P, Durieux P: Determinants of success for computerized clinical decision support systems integrated in CPOE systems: a systematic review.

    AMIA Annu Symp Proc 2006, 594-598. OpenURL

  32. Oren E, Shaffer E, Guglielmo B: Impact of emerging technologies on medication errors and adverse drug events.

    Am J Health Syst Pharm 2003, 60(14):1447-1458. OpenURL

  33. Pearson S-A, Moxey A, Robertson J, Hains I, Williamson M, Reeve J, Newby D: Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990–2007).

    BMC Health Serv Res 2009, 9(1):154. OpenURL

  34. Radley DC, Wasserman MR, Olsho LE, Shoemaker SJ, Spranca MD, Bradshaw B: Reduction in medication errors in hospitals due to adoption of computerized provider order entry systems.

    J Am Med Inform Assoc 2013, 20(3):470-6.

    doi:10.1136/amiajnl-2012-001241. Epub 2013 Feb 20

    OpenURL

  35. Reckmann M, Westbrook J, Koh Y, Lo C, Day R: Does computerized provider order entry reduce prescribing errors for hospital inpatients? A systematic review.

    J Am Med Inform Assoc 2009, 16(5):613-623. OpenURL

  36. Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW, Avery AJ: What evidence supports the use of computerized alerts and prompts to improve clinicians’ prescribing behavior?

    J Am Med Inform Assoc 2009, 16(4):531-538. OpenURL

  37. Shamliyan TA, Duval S, Du J, Kane RL: Just what the doctor ordered: review of the evidence of the impact of computerized physician order entry system on medication errors.

    Health Serv Res 2008, 43(1 Pt 1):32-53. OpenURL

  38. Shekelle P, Morton SC, Keeler EB: Costs and Benefits of Health Information Technology. Rockville (MD): Agency for Healthcare Research and Quality (US); 2006.

    (Evidence Reports/Technology Assessments, No. 132.) Available from: http://www.ncbi.nlm.nih.gov/books/NBK37988/ webcite

    OpenURL

  39. Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J: Effect of point-of-care computer reminders on physician behaviour: a systematic review.

    Can Med Assoc J 2010, 182(5):E216-E225. OpenURL

  40. Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J: The effects of on-screen, point of care computer reminders on processes and outcomes of care.

    Cochrane Database Syst Rev 2009, 3:CD001096. OpenURL

  41. van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts ACG, Bollen CW: The effect of computerized physician order entry on medication prescription errors and clinical outcome in pediatric and intensive care: a systematic review.

    Pediatrics 2009, 123(4):1184-1190. OpenURL

  42. Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, Rochon PA: The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review.

    J Gen Intern Med 2008, 23(4):451-458. OpenURL

  43. Deeks JJ, Higgins JPT, Altman DG: Chapter 9: Analysing data and undertaking meta-analyses.

    In Cochrane Handbook for Systematic Reviews of Interventions. Version 5.0.1 [updated September 2008]. The Cochrane Collaboration, 2008 Edited by Higgins JPT, Green S.

    Available from http://hiv.cochrane.org/sites/hiv.cochrane.org/files/uploads/Ch09_Analysing.pdf webcite, last accessed June 1, 2014

    OpenURL

  44. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.

    PLoS Med 2009, 6(7):e1000097. OpenURL

  45. Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M: AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.

    J Clin Epidemiol 2009, 62(10):1013-1020. OpenURL

  46. Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL: Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities.

    Am J Health Syst Pharm 2002, 59(5):436-446. OpenURL

  47. Centers for Disease Control and Prevention:

    National Hospital Discharge Survey: Centers for disease control and prevention: national hospital discharge survey. 2007.

    cited 2011 July 14]; Available from: http://www.cdc.gov/nchs/nhds/nhds_tables.htm webcite

    OpenURL

  48. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, Goldmann DA: Medication errors and adverse drug events in pediatric inpatients.

    JAMA 2001, 285(16):2114-2120. OpenURL

  49. Assessing the Evidence for Context-Sensitive Effectiveness and Safety: Developing Criteria. Rockville, MD: Agency for Healthcare Research and Quality; 2010.

    http://www.ahrq.gov/research/findings/final-reports/contextsensitive/index.html webcite

    OpenURL

  50. Franklin BD, Birch S, Savage I, Wong I, Woloshynowych M, Jacklin A, Barber N: Methodological variability in detecting prescribing errors and consequences for the evaluation of interventions.

    Pharmacoepidemiol Drug Saf 2009, 18(11):992-999. OpenURL

  51. Donyai P, O’Grady K, Jacklin A, Barber N, Franklin BD: The effects of electronic prescribing on the quality of prescribing.

    Br J Clin Pharmacol 2007, 65(2):230-237. OpenURL

  52. Franklin BD, O’Grady K, Donya P, Jacklin A, Barber N: The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: a before-and-after study.

    Qual Saf Health Care 2007, 16:279-284. OpenURL

  53. Shawahna R, Rahman N-U, Ahmad M, Debray M, Yliperttula M, Decleves X: Electronic prescribing reduces prescribing error in public hospitals.

    J Clin Nurs 2011, 20(21–22):3233-3245. OpenURL

  54. Shulman R, Singer M, Goldstone J, Bellingan G: Medication errors: a prospective cohort study of hand-written and computerised physician order entry in the intensive care unit.

    Crit Care 2005, 9(5):R516-R521. OpenURL

  55. Wess ML, Embi PJ, Besier JL, Lowry CH, Anderson PF, Besier JC, Thelen G, Hegner C: Effect of a computerized provider order entry (CPOE) system on medication orders at a community hospital and university hospital.

    AMIA Annu Symp Proc 2007, 796-800. OpenURL

  56. ITN:

    Imaging technology news: health system gets the ‘LastWord’ with GE’s centricity October 9, 2006 [cited 2013 August 14].

    Available from: http://www.itnonline.com/article/health-system-gets-%E2%80%9Clastword%E2%80%9D-ge%E2%80%99s-centricity webcite

    OpenURL

  57. Taylor R, Manzo J, Sinnett M: Quantifying value for physician order-entry systems: a balance of cost and quality.

    Healthc Financ Manage 2002, 56(7):44-48. OpenURL

  58. Barron W, Reed R, Forsythe S, Hecht D, Glen J, Murphy B, Lach R, Flores S, Tu J, Concklin M: Implementing computerized provider order entry with an existing clinical information system.

    Jt Comm J Qual Patient Saf 2006, 32(9):506-516. OpenURL

  59. Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, Vander Vliet M, Seger DL: Effect of computerized physician order entry and a team intervention on prevention of serious medication errors.

    JAMA 1998, 280(15):1311-1316. OpenURL

  60. van Doormaal JE, van den Bemt PMLA, Zaal RJ, Egberts ACG, Lenderink BW, Kosterink JGW, Haaijer-Ruskamp FM, Mol PGM: The influence that electronic prescribing has on medication errors and preventable adverse drug events: an interrupted time-series study.

    J Am Med Inform Assoc 2009, 16(6):816-825. OpenURL

  61. Westbrook J, Reckmann M, Li L, Runciman W, Burke R, Lo C, Baysari M, Braithwaite J, Day R: Effects of two commercial electronic prescribing systems on prescribing error rates in hospital in-patients: a before and after study.

    PLoS Med 2012, 9(1):e1001164. OpenURL

  62. Weant KA, Cook AM, Armitstead JA: Medication-error reporting and pharmacy resident experience during implementation of computerized prescriber order entry.

    Am J Health Syst Pharm 2007, 64(5):526-530. OpenURL

  63. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma’luf N, Boyle D, Leape L: The impact of computerized physician order entry on medication error prevention.

    JAMIA 1999, 6(4):313-321. OpenURL

  64. Colpaert K, Claus B, Somers A, Vandewoude K, Robays H, Decruyenaere J: Impact of computerized physician order entry on medication prescription errors in the intensive care unit: a controlled cross-sectional trial.

    Crit Care 2006, 10(1):R21. OpenURL

  65. Mahoney CD, Berard-Collins CM, Coleman R, Amaral JF, Cotter CM: Effects of an integrated clinical information system on medication safety in a multi-hospital setting.

    Am J Health Syst Pharm 2007, 64(18):1969-1977. OpenURL

  66. Oliven A, Michalake I, Zalman D, Dorman E, Yeshurun D, Odeh M: Prevention of prescription errors by computerized, on-line surveillance of drug order entry.

    Int J Med Inform 2005, 74:377-386. OpenURL

  67. Igboechi CA, Ng CL, Yang CS, Buckner AN: Impact of computerized prescriber order entry on medication errors at an acute tertiary care hospital.

    Hosp Pharm 2003, 38(3):227-231. OpenURL

  68. Aronsky D, Johnston PE, Jenkins G, Waitman LR, Frelix D, Jones I, Patel NR: The effect of implementing computerized provider order entry on medication prescribing errors in an emergency department.

    AMIA Annu Symp Proc 2007. OpenURL

  69. Miller RA, Waitman LR, Chen S, Rosenbloom ST: The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt.

    J Biomed Inform 2005, 38(6):469-485. OpenURL

  70. Menendez MD, Alonso J, Rancaño I, Corte JJ, Herranz V, Vázquez F: Impact of computerized physician order entry on medication errors. Elsevier Editorial System(tm) for Revista de Calidad Asistencial; 2012:16. OpenURL

  71. Wright A, Sittig DF, Ash JS, Sharma S, Pang JE, Middleton B: Clinical decision support capabilities of commercially-available clinical information systems.

    J Am Med Inform Assoc 2009, 16(5):637-644. OpenURL

  72. Shekelle PG, Pronovost PJ, Wachter RM Rockville, MD: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services. 540 Gaither Road, Rockville; MD 208502010.

    Assessing the evidence for context-sensitive effectiveness and safety of patient safety practices: developing criteria, http://www.ahrq.gov webcitecontract no. HHSA-290-2009-10001C

  73. Ogrinc G, Mooney S, Estrada C, Foster T, Goldmann D, Hall L, Huizinga M, Liu S, Mills P, Neily J, Nelson W, Pronovos PJ, Provost L, Rubenstein L, Speroff T, Splaine M, Thomson R, Tomolo A, Watts B: The SQUIRE (Standards for QUality improvement reporting excellence) guidelines for quality improvement reporting: explanation and elaboration.

    Qual Saf Health Care 2008, 17(1):i13-32. OpenURL

  74. DerSimonian R, Laird N: Meta-analysis in clinical trials.

    Control Clin Trials 1986, 7(3):177-188. OpenURL

  75. Higgins JP, Thompson SG, Deeks JJ, Altman DG: Measuring inconsistency in meta-analyses.

    BMJ 2003, 327(7414):557-560. OpenURL

  76. Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR: Empirical assessment of effect of publication bias on meta-analyses.

    BMJ 2000, 320(7249):1574-1577. OpenURL

  77. Shekelle PG, Pronovost PJ, Wachter RM, Taylor SL, Dy SM, Foy R, Hempel S, McDonald KM, Ovretveit J, Rubenstein LV, Adams AS, Angood PB, Bates DW, Bickman L, Carayon P, Donaldson L, Duan N, Farley DO, Greenhalgh T, Haughom J, Lake ET, Lilford R, Lohr KN, Meyer GS, Miller MR, Neuhauser DV, Ryan G, Saint S, Shojania KG, Shortell SM, Stevens DP, Walshe K: Advancing the science of patient safety.

    Ann Intern Med 2011, 154(10):693-696. OpenURL

  78. Wears RL, Berg M: Computer technology and clinical work: still waiting for Godot.

    JAMA 2005, 293(10):1261-1263. OpenURL