William A. Hyman
Professor Emeritus, Biomedical Engineering
Texas A&M University, w-hyman@tamu.edu
Read other articles by this author
The premise of Artificial Intelligence (AI) is that a properly constructed computer program can produce useful conclusions, and make recommendations, when presented with a fact set for a particular case (ie facts pertaining to a particular patient). I emphasize here that it is a program that does this, not “the computer”, and that this program has to be created for the particular situation at hand. It is also best if this program is well constructed so that it reaches the correct answer in a high percentage of cases. Part of being correct is identifying the population to whom the program applies, as defined by the attributes of a particular set of patient parameters. Of course, any set of patient parameters is only an abstract of the large numbers of parameters that distinguish us and our conditions, so it is good to also know if there conditions which put a patient outside the scope of the program.
AI programs are taught to do this sorting in one of two basic ways. The first is based on converting underlying knowledge into a software code that reflects that knowledge. We might call this knowledge based AI. If properly programmed, this code would reach the same conclusion as a sufficiently knowledgeable expert who knew the same underlying science, and provided that the science supported such a conclusion. This type of AI can also be made to be transparent such that the user could, if supplied with the right references, and given sufficient time under the clinical circumstances, see exactly how the program used the patient specific data to reach the conclusions that it did. Note that the sufficient time proviso is necessary in part so that such review can actually have a chance to occur.
The second type of AI program “learns” the answers to a problem (eg by “neural nets”, or other “machine learning”) by being given many case examples of parameter inputs along with the answer for each of those sets of inputs. The program then creates and memorizes the associated correlations between the input data and the already reached conclusions. After sufficient training the program could be presented with new cases and based on the cases the program was trained with, the program would, if possible, fit the case into its learned patterns. In such a system there is no underlying knowledge or understanding, instead the program has been taught to memorize correlations, using software designed for this purpose. We should call this Artificial Memorization (AM) rather than Artificial Intelligence. In this type of system there is no transparency and the user cannot trace the logic by which the program reaches a conclusion because the user could not, even if they had access, memorize all of the presumably large set of specific cases from which the system learned.
The transparency distinction was included in the 21st Century Cures Act which deregulated certain Clinical Decision Support (CDS) software if the professional user could conduct an independent review of the results. However, the FDA was at the same time given authority to include or retain in its regulatory portfolio any software that would be reasonably likely to have serious adverse health consequences. How the FDA will implement these CDS provisions in an open and consistent fashion is still awaiting clarification, perhaps in the form of a now long awaited CDS Guidance. But curiously such a Guidance has dropped off the FDA’s priority work list.
By way of analogy, consider someone who can memorize a set of data, as opposed to someone who understands the data in the broad context of the scientific literature. The memorizer can tell you the data but offers no insight, and querying them beyond the data itself would not be illuminating. None-the-less accessing the data through the memorizer might have some value in some cases. The knowledge based expert can tell you, based on the understanding they have acquired, what a newly presented situation (inputs) likely means. The user of the knowledge based result can be second guessed by accessing the same underlying knowledge that was used to generate the expert’s understanding, or by having the expert explain their reasoning.