The Dichotomy of Health Information Technology
William Hersh, MD, Professor and Chair, OHSU
Blog: Informatics Professor
I often get involved in debates about the value of health information technology (HIT) interventions in healthcare. While the optimist in me likes to point to the growing body of scientific evidence showing efficacy, the realist in me takes seriously the negative outcomes that some studies as well as reported experiences show. This leads to a question that some may ask, which is why does there exist this apparent dichotomy of scientific evidence supporting the use of HIT in the face of widespread dissatisfaction with it in many settings?
A number of “negative” studies have appeared in recent months [1, 2], although these studies have some significant methodologic limitations that I will describe further below. In addition, the scientific basis for use of HIT remains strong. Systematic reviews in recent years have concluded its value, whether approached from the standpoint of clinical outcomes [3] or meaningful use criteria [4]. Nonetheless, there is widespread dissatisfaction among many users of HIT, especially physicians, as exemplified in a couple surveys published by the magazine Medical Economics last year [5]. The advocacy of esteemed groups such as the Institute of Medicine for more study and regulation around HIT safety demonstrates that such problems are real [6].
While some in the informatics field point to more nefarious reasons for this apparent dichotomy, such as financial motivations by those who stand to benefit, i.e., EHR vendors, I believe that HIT has a fundamental difficulty in translating efficacy into effectiveness. The difference between efficacy and effectiveness is a well-known concept in clinical epidemiology, and is best demonstrated that some clinical interventions (tests, treatments, etc.) work well in highly controlled settings, such as well-resourced academic medical centers or when limited to patient populations that lack co-morbid conditions that most patients in the healthcare system typically have [7].
It is also worthwhile to delve further into the methodology of some of these negative studies, especially in the current highly charged political environment around HIT, including its role in healthcare reform. Take the study of Samal et al. [1]. This investigation compared the quality of care as measured by performance on mostly process-based quality measures in a single organization between physicians who achieved Stage 1 of meaningful use vs. those who did not. There are all sorts of issues whether quality measures unrelated to an EHR intervention are a good measure of an EHR system’s value. There is also an inconsistent relationship between performance on quality measures and patient outcomes from care [8].
The study by McDonald et al. surveyed internal medicine physicians about various aspects of EHR use, such as whether it added or diminished free time [2]. Nearly 60% of respondents indicated EHR use reduced free time by an average of 77.5 minutes per day. Although many other variables were assessed, such as EHR vendor as well as practice size and setting, there was no analysis of which of these factors may have impacted free time. In particular, it would be interesting to compare the 60% who reported losing time with the 15% who said EHRs made them more efficient and the 26% who said that the time change was neutral. What was it about the physicians who did not lose time with their EHRs that made them different from their colleagues who claimed lost time? Was it their vendor? Or their practice situation or size? Or maybe even the availability of clinical informatics expertise guiding them.
Another concern about this study is that it was a recall-based survey. What would have been more useful was the use of real time-motion studies. These have been done in the past, and the added time is minimal [9]. It would also have been good to ask these physicians if they wanted to return to the days of paper records, with their illegibility, inaccessibility, and other problems.
I am in no way arguing that negative studies of EHR should be discounted. But like all areas of scientific study, we must weigh all the evidence. It is clear that a major challenge to HIT is how to translate efficacy into effectiveness. This requires research looking at why its benefits are not readily generalizable to different settings. Such studies need to assess all possible factors, from healthcare setting type to physician characteristics to the availability of suitable informatics expertise. We must also not lose sight of what we are trying to improve with HIT, namely a healthcare system that is unsafe, wasteful, and achieves suboptimal outcomes [10].
References
1. Samal, L, Wright, A, et al. (2014). Meaningful use and quality of care. JAMA Internal Medicine. 174: 997-998.
2. McDonald, CJ, Callaghan, RM, et al. (2014). Use of internist’s free time by ambulatory care electronic medical record systems. JAMA Internal Medicine: Epub ahead of print.
3. Buntin, MB, Burke, MF, et al. (2011). The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Affairs. 30: 464-471.
4. Jones, SS, Rudin, RS, et al. (2014). Health information technology: an updated systematic review with a focus on meaningful use. Annals of Internal Medicine. 160: 48-54.
5. Verdon, DR (2014). Physician outcry on EHR functionality, cost will shake the health information technology sector. Medical Economics, February 10, 2014. http://medicaleconomics.modernmedicine.com/medical-economics/news/physician-outcry-ehr-functionality-cost-will-shake-health-information-technol.
6. Anonymous (2012). Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC, National Academies Press.
7. Singal, AG, Higgins, PDR, et al. (2014). A primer on effectiveness and efficacy trials. Clinical and Translational Gastroenterology. 5: e45. http://www.nature.com/ctg/journal/v5/n1/full/ctg201313a.html.
8. Houle, SK, McAlister, FA, et al. (2012). Does performance-based remuneration for individual health care practitioners affect patient care?: a systematic review. Annals of Internal Medicine. 157: 889-899.
9. Overhage, JM, Perkins, S, et al. (2001). Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory primary care internal medicine practices. Journal of the American Medical Informatics Association. 8: 361-371.
9. Smith, M, Saunders, R, et al. (2012). Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC, National Academies Press.
This article post first appeared on The Informatics Professor. Dr. Hersh is a frequent contributing expert to HITECH Answers.