The future of AI in the NHS - part four

Earlier this year, I completed the ‘Artificial Intelligence in Health Care’ course at the MIT Sloan School of Management. I found it to be fascinating, thought provoking and very worthwhile. Specifically, it took a subject I was generally interested in and helped me focus on the precise issues of relevance to not only my practice but the future of NHS health care overall.

In this series of blog posts, I am repurposing my written assignments as articles so colleagues in the NHS and globally can share in that thought process. You will see that, at this stage, there are more questions than answers, but hopefully together we can work to move the theoretical towards the practical.

What are the benefits and challenges of interpretability in machine learning?

Following on from the previous article, we are now moving into controversial areas of dealing with AI in general, and not necessarily restricted to the healthcare domain, such as trust and consistency.

Before I proceed to propose questions relevant to interpretability in machine learning, I would like to reflect on the matters of trust and consistency when it comes to healthcare, as they are related to the human healthcare providers rather than AI itself. As a physician, I instinctively deal with these issues daily. The entire doctor-patient relationship is based on trust, and the credibility of the physician is partially derived from consistency in delivering in the same level of care for all patients in all circumstances. In fact, it would be interesting to research whether there are studies that compare AI to humans in terms of trust, consistency and explainability.

The NHS, combined with the social care sector, serves the entire British population (approximately 67 million). NHS digital services utilize the vast data set of the NHS to train AI before deployment in the field. This model contrasts with others around the world, such as in the US, where there is far more than one entity that generate and use data set for AI machine learning. The UK model has risks and rewards; one risk is that if a data bias is introduced into the algorithm, that error can propagate very quickly and result in less-than-reliable outcome. This, in turn, can negatively impact trust in AI.

I discussed previously how the public and some healthcare professionals have been pushing back against AI in healthcare. A lack of interpretability and consistency will only augment such doubts. A common argument is that AI is perceived as a “black box”, with no visibility of what goes on inside it, and with no accountability to explain the outcomes it produces. I believe data bias, and subsequent training algorithms, is a fundamental issue to deal with to promote a consistent and reliable outcome. When such an outcome is proven repeatedly, there will be less pushback and, in time, trusting AI will become inherent (though there should always be human oversight).

Based on the above, I propose the following questions and launching points for further discussion:

1. Which model of interpretability should be followed when designing machine learning algorithms in healthcare? While a model that is interpretable by default seems to be a logical choice, it is too simple and linear. Therefore, it would not be suitable to handle complex data with large parameters and variables. Another approach is retrofitting interpretability to existing complex AI model that already handle complex data and are perceived as black boxes. This method is likely to carry the risk of errors and not improve trust in the notion of the black box. This is especially true when the model is complex and impacts a large portion of the population. Finally, there is the possibility of building the mechanism within the model itself. Although such a solution is quite involved and complex, it is likely to yield a better outcome, as the AI solution provides responses for its actions and results when interpreted. Such a solution would also enhance the credibility and trust in AI when used in healthcare. The analogy for such enhancement of trust is an organization known as the National Institute of Clinical Excellence (NICE), which oversees literature reviews, clinical research and appraising of clinical evidence before it produces national guidelines for British healthcare professionals to follow. There is general acceptance and trust in NICE guidelines, therefore a good basis for establishing a similar organization responsible for interpretability of AI to confirm to the healthcare professionals and the public that the AI-provided outcome is consistent and trustworthy. I appreciate that building interpretability into AI algorithms can be complex, costly and time intensive. The NHS is currently pursuing the first steps in introducing AI therefore, it is likely to be much easier to undertake that option now rather than retrofitting a black box later.
   

2. How can errors in the algorithmic interpretation of data from varying and new sources be prevented in a healthcare system like the NHS, that relies on vast and varied hardware infrastructure? I explained in one of my previous posts that the NHS is subject to centralized and convoluted management processes. Additionally, it is also the victim of political and financial restrictions. It is a massive organization that has been an existence for over 70 years and its infrastructure has significant regional variation in addition to an aging technology which must be constantly updated. I began to wonder how the NHS IT infrastructure will manage the complexity and resources required by AI across the entire sector of healthcare. I concluded that it will be a very challenging undertaking that will require scientific, financial and political consensus to ensure homogenous distribution of resources in order to provide consistent and reliable service throughout the whole system. At present, it is unclear if there is such a will or entity to undertake this task.
   

3. As we advance our knowledge in and use of AI in healthcare, is it possible that our ability to enforce AI interpretability and explainability will be exceeded? If so, can this be prevented? Advances in AI are moving at an alarming rate. This includes advancements in healthcare data interpretation, diagnostics and therapeutics, although the speed of advancement may not conform completely to Moore’s law, it is pursuing a similar trajectory. Therefore, it is not unrealistic to expect that there will soon be an inflection point where we cross the threshold of our ability to enforce interpretability and explainability of AI. Should we consider implementing tools and guard rails around such an issue rather than waiting until it is a reality. I am aware of recent governmental efforts to put safety nets around AI. This is a first positive step, but I believe it will require global collaboration at the United Nations or WHO level.
   

4. In the UK, like the USA and other advanced economies, the societal demographic structure is dynamic, so how do we ensure that such dynamism is updated in the AI training algorithms to mitigate racial biases in the resultant data? I believe building parameters into an AI model to accommodate for different racial ethnicities is perhaps less challenging than the issues discussed above. However, the recent significant migration and immigration movements over the last 50 to 70 years can impact prevalence and incidence of illnesses. This, in turn, will impact on our accumulated knowledge of illnesses and medical literature. When using a data set to train AI models, we should consider adjusting it to take into account the changes in demographics and racial distribution. This will hopefully reduce racial data bias and improve trust and reliability of AI in healthcare.

Do you have any thoughts about what I’ve outlined here, or further questions to contribute? Please connect on social media and let’s talk!

Connect with Contempo and join the discussion!