The classic computer science adage “garbage in, garbage out” lacks nuance in terms of understanding biased medical data, argue computer science and bioethics professors from MIT, Johns Hopkins University, and the Alan Turing Institute in a latest opinion piece published in a recent edition of the . The rising popularity of artificial intelligence has brought increased scrutiny to the matter of biased AI models leading to algorithmic discrimination, which the White House Office of Science and Technology identified as a key issue of their recent Blueprint for an AI Bill of Rights.
When encountering biased data, particularly for AI models utilized in medical settings, the everyday response is to either collect more data from underrepresented groups or generate synthetic data making up for missing parts to make sure that the model performs equally well across an array of patient populations. However the authors argue that this technical approach needs to be augmented with a sociotechnical perspective that takes each historical and current social aspects under consideration. By doing so, researchers will be more practical in addressing bias in public health.
“The three of us had been discussing the ways wherein we frequently treat issues with data from a machine learning perspective as irritations that must be managed with a technical solution,” recalls co-author Marzyeh Ghassemi, an assistant professor in electrical engineering and computer science and an affiliate of the Abdul Latif Jameel Clinic for Machine Learning in Health (Jameel Clinic), the Computer Science and Artificial Intelligence Laboratory (CSAIL), and Institute of Medical Engineering and Science (IMES). “We had used analogies of knowledge as an artifact that provides a partial view of past practices, or a cracked mirror holding up a mirrored image. In each cases the data is probably not entirely accurate or favorable: Possibly we predict that we behave in certain ways as a society — but while you actually have a look at the information, it tells a distinct story. We won’t like what that story is, but when you unearth an understanding of the past you possibly can move forward and take steps to deal with poor practices.”
Data as artifact
Within the paper, titled “Considering Biased Data as Informative Artifacts in AI-Assisted Health Care,” Ghassemi, Kadija Ferryman, and Maxine Mackintosh make the case for viewing biased clinical data as “artifacts” in the identical way anthropologists or archeologists would view physical objects: pieces of civilization-revealing practices, belief systems, and cultural values — within the case of the paper, specifically those who have led to existing inequities within the health care system.
For instance, a 2019 study showed that an algorithm widely considered to be an industry standard used health-care expenditures as an indicator of need, resulting in the erroneous conclusion that sicker Black patients require the identical level of care as healthier white patients. What researchers found was algorithmic discrimination failing to account for unequal access to care.
On this instance, quite than viewing biased datasets or lack of knowledge as problems that only require disposal or fixing, Ghassemi and her colleagues recommend the “artifacts” approach as a technique to raise awareness around social and historical elements influencing how data are collected and alternative approaches to clinical AI development.
“If the goal of your model is deployment in a clinical setting, you must engage a bioethicist or a clinician with appropriate training reasonably early on in problem formulation,” says Ghassemi. “As computer scientists, we frequently don’t have a whole picture of different social and historical aspects which have gone into creating data that we’ll be using. We’d like expertise in discerning when models generalized from existing data may not work well for specific subgroups.”
When more data can actually harm performance
The authors acknowledge that one in all the more difficult facets of implementing an artifact-based approach is having the ability to assess whether data have been racially corrected: i.e., using white, male bodies as the traditional standard that other bodies are measured against. The opinion piece cites an example from the Chronic Kidney Disease Collaboration in 2021, which developed a latest equation to measure kidney function since the old equation had previously been “corrected” under the blanket assumption that Black people have higher muscle mass. Ghassemi says that researchers needs to be prepared to analyze race-based correction as a part of the research process.
In one other recent paper accepted to this yr’s International Conference on Machine Learning co-authored by Ghassemi’s PhD student Vinith Suriyakumar and University of California at San Diego Assistant Professor Berk Ustun, the researchers found that assuming the inclusion of personalized attributes like self-reported race improve the performance of ML models can actually result in worse risk scores, models, and metrics for minority and minoritized populations.
“There’s no single right solution for whether or not to incorporate self-reported race in a clinical risk rating. Self-reported race is a social construct that’s each a proxy for other information, and deeply proxied itself in other medical data. The answer needs to suit the evidence,” explains Ghassemi.
The way to move forward
This isn’t to say that biased datasets needs to be enshrined, or biased algorithms don’t require fixing — quality training data continues to be key to developing secure, high-performance clinical AI models, and the piece highlights the role of the National Institutes of Health (NIH) in driving ethical practices.
“Generating high-quality, ethically sourced datasets is crucial for enabling the usage of next-generation AI technologies that transform how we do research,” NIH acting director Lawrence Tabak stated in a press release when the NIH announced its $130 million Bridge2AI Program last yr. Ghassemi agrees, declaring that the NIH has “prioritized data collection in ethical ways in which cover information now we have not previously emphasized the worth of in human health — corresponding to environmental aspects and social determinants. I’m very enthusiastic about their prioritization of, and powerful investments towards, achieving meaningful health outcomes.”
Elaine Nsoesie, an associate professor on the Boston University of Public Health, believes there are numerous potential advantages to treating biased datasets as artifacts quite than garbage, starting with the concentrate on context. “Biases present in a dataset collected for lung cancer patients in a hospital in Uganda is likely to be different from a dataset collected within the U.S. for a similar patient population,” she explains. “In considering local context, we can train algorithms to raised serve specific populations.” Nsoesie says that understanding the historical and contemporary aspects shaping a dataset could make it easier to discover discriminatory practices that is likely to be coded in algorithms or systems in ways in which should not immediately obvious. She also notes that an artifact-based approach could lead on to the event of recent policies and structures ensuring that the basis causes of bias in a selected dataset are eliminated.
“People often tell me that they’re very afraid of AI, especially in health. They’ll say, ‘I’m really fearful of an AI misdiagnosing me,’ or ‘I’m concerned it would treat me poorly,’” Ghassemi says. “I tell them, you mustn’t be fearful of some hypothetical AI in health tomorrow, try to be fearful of what health is immediately. If we take a narrow technical view of the information we extract from systems, we could naively replicate poor practices. That’s not the one option — realizing there’s an issue is our first step towards a bigger opportunity.”