New evaluation sets a “starting point” for raising standards to improve EHR design and user-testing
In an effort to promote transparency around how electronic health records (EHRs) are designed and user-tested, and drive improvements in clinician support and patient safety, the American Medical Association (AMA) and MedStar Health’s National Center for Human Factors in Healthcare have developed a comparative EHR User-Centered Design Evaluation Framework that shows a lack of focus among regulators and industry on user-centered design and usability testing.
The EHR User-Centered Design Evaluation Framework drills down on the design and testing processes for optimizing usability in 20 common EHR products using information reported by EHR vendors to meet usability certification requirements set by the Office of the National Coordinator for Health Information Technology (ONC). The framework goes beyond the low gauge set by federal usability certification criteria and evaluates the vendor’s compliance with best practices for user-centered design (UCD) processes according to a 15 point scale developed by MedStar’s Human Factors Center in collaboration with the AMA.
“It is important to recognize that this framework evaluates conformity with best practices identified in human factors and usability literature for user-centered design and testing. We are not evaluating the actual usability of the product as experienced by end users,” said Raj Ratwani, Ph.D., scientific director of the Human Factors Center and a principle developer of the framework. “Alignment with best practices for user-centered design and testing is a starting point that regulators and industry should meet and exceed. The framework we developed is the first step in bringing greater transparency to the usability processes of EHR vendors.”
Comprehensive user-centered design and testing for EHR products is essential for ensuring that systems are useful clinical tools that will help improve patient safety and meet the cognitive workflow and information needs of physicians and other health professionals. This is emphasized by the AMA and MedStar Health in the goals of the framework, which state: “We believe EHRs should be designed with the end user in mind and that the ONC’s requirements do not go far enough to encourage fully functional and usable products. This framework can be used by the ONC to improve their certification program, and as a method to track improvements EHR vendors make as they recertify their products over time.”
The ONC requires vendors to report on the UCD processes they followed for only a handful of EHR capabilities. Since the design and testing of most EHR functions goes unverified by the ONC’s certification criteria, an optimal evaluation presented by the framework should not be considered reflective of actual usability experienced by physicians and other end-users. As such, the framework should not be interpreted as a buyer’s guide.
“Physician experiences documented by the AMA demonstrate that most EHR systems fail to support effective and efficient clinical work, and continued issues with usability are a key factor driving low satisfaction with many EHR products,” said AMA President Steven J. Stack, M.D. “Our goal is to shine light on the low-bar of the certification process and how EHRs are designed and user-tested in order to drive improvements that respond to the urgent physician need for better designed EHR systems.”
Ratwani added, “To improve the usability of EHRs we need to promote rigorous usability development processes based on recognized methods and standards.”
The framework builds on a study Ratwani wrote on EHR usability that was published by the Journal of the American Medical Association in September 2015. In that study, Ratwani, et al, analyzed available reports from the top 50 EHR vendors, as measured by the number of meaningful use attestations made to the ONC between April 1, 2013, and Nov. 30, 2014. The researchers found that a significant percentage of vendors did not even follow basic federal requirements for certifying the usability of their products, yet these products were certified under the meaningful-use incentive program.
Effective usability is critical to patient safety and physician satisfaction. A study conducted by the AMA and RAND Corporation in 2013 found that EHR usability “represents a unique and vexing challenge to physician professional satisfaction” and that the current state of technology raises “concerns about effects on patient care.” The AMA-RAND report urged improved usability among federal certification criteria.
The AMA will continue to move aggressively to advance the goal of more usable EHR products based on eight usability priorities created last year by the AMA to lead EHR design improvements that would ultimately improve the health of the nation.