Organizations are increasingly utilizing machine-learning models to allocate scarce resources or opportunities. For example, such models will help corporations screen resumes to decide on job interview candidates or aid hospitals in rating kidney transplant patients based on their likelihood of survival.
When deploying a model, users typically strive to make sure its predictions are fair by reducing bias. This often involves techniques like adjusting the contains a model uses to make decisions or calibrating the scores it generates.
Nonetheless, researchers from MIT and Northeastern University argue that these fairness methods should not sufficient to deal with structural injustices and inherent uncertainties. In a latest paper, they show how randomizing a model’s decisions in a structured way can improve fairness in certain situations.
For instance, if multiple corporations use the identical machine-learning model to rank job interview candidates deterministically — with none randomization — then one deserving individual might be the bottom-ranked candidate for each job, perhaps attributable to how the model weighs answers provided in a web-based form. Introducing randomization right into a model’s decisions could prevent one worthy person or group from at all times being denied a scarce resource, like a job interview.
Through their evaluation, the researchers found that randomization might be especially useful when a model’s decisions involve uncertainty or when the identical group consistently receives negative decisions.
They present a framework one could use to introduce a certain amount of randomization right into a model’s decisions by allocating resources through a weighted lottery. This method, which a person can tailor to suit their situation, can improve fairness without hurting the efficiency or accuracy of a model.
“Even should you could make fair predictions, must you be deciding these social allocations of scarce resources or opportunities strictly off scores or rankings? As things scale, and we see an increasing number of opportunities being decided by these algorithms, the inherent uncertainties in these scores might be amplified. We show that fairness may require some form of randomization,” says Shomik Jain, a graduate student within the Institute for Data, Systems, and Society (IDSS) and lead writer of the paper.
Jain is joined on the paper by Kathleen Creel, assistant professor of philosophy and computer science at Northeastern University; and senior writer Ashia Wilson, the Lister Brothers Profession Development Professor within the Department of Electrical Engineering and Computer Science and a principal investigator within the Laboratory for Information and Decision Systems (LIDS). The research shall be presented on the International Conference on Machine Learning.
Considering claims
This work builds off a previous paper during which the researchers explored harms that may occur when one uses deterministic systems at scale. They found that using a machine-learning model to deterministically allocate resources can amplify inequalities that exist in training data, which may reinforce bias and systemic inequality.
“Randomization is a really useful concept in statistics, and to our delight, satisfies the fairness demands coming from each a systemic and individual viewpoint,” Wilson says.
In this paper, they explored the query of when randomization can improve fairness. They framed their evaluation across the ideas of philosopher John Broome, who wrote concerning the value of using lotteries to award scarce resources in a way that honors all claims of people.
An individual’s claim to a scarce resource, like a kidney transplant, can stem from merit, deservingness, or need. For example, everyone has a right to life, and their claims on a kidney transplant may stem from that right, Wilson explains.
“Whenever you acknowledge that individuals have different claims to those scarce resources, fairness goes to require that we respect all claims of people. If we at all times give someone with a stronger claim the resource, is that fair?” Jain says.
That form of deterministic allocation could cause systemic exclusion or exacerbate patterned inequality, which occurs when receiving one allocation increases a person’s likelihood of receiving future allocations. As well as, machine-learning models could make mistakes, and a deterministic approach could cause the identical mistake to be repeated.
Randomization can overcome these problems, but that doesn’t mean all decisions a model makes needs to be randomized equally.
Structured randomization
The researchers use a weighted lottery to regulate the extent of randomization based on the quantity of uncertainty involved within the model’s decision-making. A choice that’s less certain should incorporate more randomization.
“In kidney allocation, often the planning is around projected lifespan, and that’s deeply uncertain. If two patients are only five years apart, it becomes rather a lot harder to measure. We would like to leverage that level of uncertainty to tailor the randomization,” Wilson says.
The researchers used statistical uncertainty quantification methods to find out how much randomization is required in numerous situations. They show that calibrated randomization can result in fairer outcomes for people without significantly affecting the utility, or effectiveness, of the model.
“There’s a balance available between overall utility and respecting the rights of the individuals who’re receiving a scarce resource, but oftentimes the tradeoff is comparatively small,” says Wilson.
Nonetheless, the researchers emphasize there are situations where randomizing decisions wouldn’t improve fairness and will harm individuals, corresponding to in criminal justice contexts.
But there might be other areas where randomization can improve fairness, corresponding to college admissions, and the researchers plan to check other use cases in future work. Additionally they wish to explore how randomization can affect other aspects, corresponding to competition or prices, and the way it might be used to enhance the robustness of machine-learning models.
“We hope our paper is a primary move toward illustrating that there may be a profit to randomization. We’re offering randomization as a tool. How much you will wish to do it’ll be as much as all of the stakeholders within the allocation to choose. And, in fact, how they determine is one other research query all together,” says Wilson.
