The Learning Healthcare System and Evidence-Based Medicine

The enthusiasm for big data and for the use of analytics and business intelligence with that data is reaching a fevered pitch. I share that enthusiasm, but also know from both my clinical and my informatics experience that knowledge will not emanate just by turning on the data spigot from the growing number of electronic health record (EHR) systems now in operational use. However, if we approach the problem properly, I believe we can achieve the goals of the learning healthcare system as eloquently laid out in various reports from the Institute of Medicine (IOM) [1, 2].

One sensible approach was published recently in Annals of Internal Medicine [3]. The authors were from Group Health Cooperative in Seattle, a leader in the use of data and information systems to improve the quality and outcomes of care. The paper is summarized well by a figure that shows a continuous cycle of design-implementation-evaluation-adjustment of improved care, with interaction with the external environment through scanning for identification of problems and solutions and dissemination to share what has been learned in their setting.

A complementary approach to learning from EHR and other clinical data can be to apply the basic approach of evidence-based medicine (EBM) [4]. In some ways, EBM is antagonistic to EHR data analytics, with the former giving the most value to evidence from controlled experiments, especially randomized controlled trials (RCTs), while the latter makes use of real-world observational data that may be incomplete, incorrect, and inconsistent.

But I maintain that we can look to the process of EBM to guide us in how to best approach the “evidence” of EHR data analytics and the learning health system. EBM is not just about finding RCTs. Rather, it uses a principled approach to find and apply the best evidence to make clinical decisions. In particular, EBM done most effectively uses four steps:

1. Ask an answerable question
2. Find the best evidence
3. Critically appraise the evidence
4. Apply it to the patient situation

When I teach EBM, I emphasize that the first step of asking an answerable question may be the most important. It is not enough, for example, to ask if a test or treatment works. Rather, we need to know at a minimum whether it works relative to some alternative approach in a particular patient population or setting. This same approach is obviously necessary in the learning health system. Just as RCTs do not inform us passively, neither will EHR data analytics approaches.

In the second step, the principle from EBM is very much the same, even if the techniques of obtaining evidence are very different. The “evidence” in the case of the learning health system is the data in EHR and other systems that, as noted above, may be incomplete, incorrect, and inconsistent. We therefore need to determine if we have the proper data and, if so, whether it can applied to answer our question.

For the third step, just as with EBM, we must critically appraise our evidence. Can we trust the inferences and conclusions from the data? Are there confounding variables of which we may not be aware? This may be critical with EHR data where assignment of cause and effect could be difficult, if not impossible. The solution likely comes back to asking the right question, i.e., one we can have confidence in the correct answer.

Finally, we have to ask, can the data be applied in our setting? Just as some RCTs answer questions in patient populations very different from those of the clinician making decisions, it must be ascertained if the results obtained from this approach can be applied to a specific patient or setting.

The growing quantity of clinical data in operational clinical systems provides a foundation for the learning healthcare system. However, we must approach the questions we ask and how we answer them with caution and a sound methodology. The approach of EBM offers a framework for carrying out this very different but complementary work.

References

1. Eden J, Wheatley B, McNeil B, and Sox H, eds. Knowing What Works in Health Care: A Roadmap for the Nation. 2008, National Academies Press: Washington, DC.
2. Smith M, Saunders R, Stuckhardt L, and McGinnis JM, Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. 2012, Washington, DC: National Academies Press.
3. Greene SM, Reid RJ, and Larson EB, Implementing the learning health system: from concept to action. Annals of Internal Medicine, 2012. 157: 207-210.
4. Straus SE, Glasziou P, Richardson WS, and Haynes RB, Evidence-Based Medicine: How to Practice and Teach It, 4e. 2010, New York, NY: Churchill Livingstone.