AI is already getting used in health care. Some hospitals use the technology to assist triage patients. Some use it to assist diagnosis, or to develop treatment plans. However the true extent of AI adoption is unclear, says Sandra Wachter, a professor of technology and regulation on the University of Oxford within the UK.

“Sometimes we don’t actually know what sorts of systems are getting used,” says Wachter. But we do know that their adoption is more likely to increase because the technology improves and as health-care systems look for methods to scale back costs, she says.

Research suggests that doctors may already be putting a whole lot of faith in these technologies. In a study published a number of years ago, oncologists were asked to match their diagnoses of skin cancer with the conclusions of an AI system. A lot of them accepted the AI’s results, even when those results contradicted their very own clinical opinion.

There’s a really real risk that we’ll come to depend on these technologies to a greater extent than we must always. And here’s where paternalism could are available in.

“Paternalism is captured by the idiom ‘the doctor knows best,’” write Melissa McCradden and Roxanne Kirsch of the Hospital for Sick Children in Ontario, Canada, in a recent scientific journal paper. The concept is that medical training makes a health care provider one of the best person to make a call for the person being treated, no matter that person’s feelings, beliefs, culture, and anything which may influence the alternatives any of us make.

“Paternalism may be recapitulated when AI is positioned as the best type of evidence, replacing the all-knowing doctor with the all-knowing AI,” McCradden and Kirsch proceed. They are saying there’s a “rising trend toward algorithmic paternalism.” This is able to be problematic for an entire host of reasons.