By Jordan Everson, Bob Phillips, and Nathaniel Hendrix, ONC
LinkedIn:Â Jordan Everson
LinkedIn: Bob Phillips, MD MSPH
LinkedIn:Â ONC
Since 2021, the Assistant Secretary for Technology Policy and Office of the National Coordinator for Health IT (hereafter ASTP) and the American Board of Family Medicine (ABFM) have collaborated to develop and field survey questions designed to better understand family physicians’ experience using health information technology (health IT) in primary care. We’ve now fielded questions for three years (2022-2024) through multiple questionnaires, and the results have driven data briefs and studies.
A Novel Collaboration to Connect Policy and Practice
Three parties worked together under this cooperative agreement: ABFM, ASTP, and the University of California, San Francisco (UCSF), drawing on diverse expertise to work towards a set of survey questions. The team at ABFM brought a deep knowledge of the benefits and frustrations that electronic health records (EHRs) and interoperability can bring to practicing physicians and more than a decade of tracking these issues in physician surveys. Our team at ASTP ensured that survey questions could be used to directly inform policy decisions, while the UCSF researchers kept the work tied closely to ongoing research on the use and impact of health IT. A new article published in the Journal of the American Medical Informatics Association (JAMIA) compares the results of various physician surveys on health IT and highlights the value, accuracy, and novelty of the data ABFM collected.
In 2022, after defining the questions, ABFM included them on the Continuous Certification Questionnaire (CCQ); the questionnaire family medicine physicians complete as they renew their board certification. As part of the cooperative agreement, UCSF included the questions on a voluntary survey fielded in 2022 to physicians of all specialties nationwide. By embedding these questions in their recertification process, ABFM achieved a 100% response rate, eliminating concerns about response bias that impact all surveys and particularly surveys of busy professionals. ABFM was also able to quickly accrue responses, as between 5% and 15% of more than 105,000 board-certified family physicians who recertify each year complete the surveys as they register.
By comparison, only 18.2% of contacted physicians responded to the National Electronic Health Records Survey (NEHRS) in 2021, a collaboration between ASTP and the Centers for Disease Control and Prevention’s National Center for Health Statistics. UCSF’s nationwide survey resulted in 3.8% of contacted physicians responding to the Physician Health Information Technology Survey (PHITS). Both surveys required months of repeated outreach to accrue responses.
As we compared the results of these surveys, we quickly realized that the respondent from each looked substantially different in terms of individual and practice characteristics. For instance, NEHRS respondents were far more likely to be in independent practice than ABFM respondents, and PHITS respondents were much more likely to work in academic centers. Yet despite the varied response rates and practice characteristics, surprisingly, when we looked at what physicians said about health IT, we saw many similar trends between the NEHRS and ABFM data. In comparison, respondents to the PHITS were substantially more negative about their health IT; perhaps because the small fraction of contacted physicians who chose to respond to this a voluntary survey were motivated by disproportionately negative views.
There was one key question on which ABFM and NEHRS respondents diverged: ABFM respondents reported substantially higher time spent documenting in their EHRs outside of work hours than did NEHRS respondents. It makes sense to us that physicians reporting especially high levels of documentation burden would be less likely to respond to voluntary surveys. This points to one of the main advantages of working with data from certifying boards – namely, that they can better capture data from those physicians who are likely the most vulnerable to poor EHR design and interoperability infrastructure.
Re-grounding the Survey for New Insights
In the third year of the cooperative agreement, we revisited the ABFM CCQ fielded in 2022 and 2023 to maximize the value of the questions we were posing to family physicians. Over the course of 20 interviews and four focus groups with family doctors and consultation with a technical expert panel, we refined survey questions to best track how doctors experience exchanging information about their patients through interoperable health IT. At the same time, we trimmed questions from the survey that were highly correlated with or partially duplicated information from other survey questions, meaning that we have decreased response burden.
The first data from those revised questions has just become available, and we are analyzing these data and eager to share new insights with our federal partners and the public.
Interested in the ABFM data? Data from the three years of this cooperative agreement (2022-2024) are available by request of ABFM at (research@theabfm.org) following ABFM’s review process.
This article was originally published on the Health IT Buzz and is syndicated here with permission.