New York, NY – The healthcare industry is rushing to adopt artificial intelligence. Just last year, U.S. health organizations spent $1.4 billion on AI tools. For doctors and other medical professionals, AI promises to make their lives easier by improving patient data and supporting decision-making, but it also raises red flags about potential biases and errors. A new Columbia Business School study reveals that using biased medical technology can have significant, long-term impacts on doctors’ decision-making. The study found that a technology used to assist doctors—manipulated to recommend prescribing certain long-acting opioid painkillers for pain treatment—influenced doctors to significantly overprescribe those opioids to patients, even for years after the doctors stopped using the technology, moved locations, and faced new state-level regulations.
The study, The Impact of Manipulated CDS Algorithm on Opioid Prescribing Decision, by Xuelin Li, Assistant Professor of Business at Columbia Business School and Boston University Professor Meizi Zhou, takes an in-depth look at a real-world example in which a major health technology company conspired with pharmaceutical companies to intentionally manipulate clinical decision support (CDS) alerts in its electronic health record (EHR) software for years to promote opioid painkillers. These alerts influenced doctors to increase prescriptions of the medication to patients. To study this, the researchers created a detailed dataset of opioid prescriptions by around 60,000 doctors between 2013 and 2021, taken from EHR software information downloaded from the HealthIT.gov website. Using a machine-learning algorithm, the researchers analyzed the amount of opioid prescriptions that individual doctors made before, during, and after the software was manipulated. Their results showed that doctors who used the biased software between 2016 and 2018 prescribed nearly 6% more long-acting opioid painkillers to patients than doctors who did not use the technology. Even after the biased algorithm was no longer in use, those same affected doctors still prescribed 11.6% more opioids compared to doctors who did not use the technology, even when the doctors moved and treated new patients, and even in areas with stricter regulations. However, the effects appear to be smaller in settings with greater physician awareness after the technological bias received significant public attention in the EHR software company’s $145 million legal settlement in 2020. This suggests that while regulations may fall short, public awareness and transparency about algorithmic biases can help mitigate their harmful effects.
As the healthcare industry rapidly adopts artificial intelligence tools, the new study raises an alarm on the long-term effects that technological biases can have, even on highly trained professionals. While AI adoption in healthcare can revolutionize the industry, the study highlights that leaders should cautiously implement the technology and encourage transparency about possible biases and errors.
This research is part of Columbia Business School’s broader efforts to advance applied artificial intelligence in healthcare safety and translate emerging technologies into real-world impact. Through its AI+Healthcare initiative, the School is focused on how data and AI can improve patient care, strengthen health systems, and expand access to care. To learn more, visit https://business.columbia.edu/ai-healthcare.
