Finding a greater way
Each time an Amsterdam resident applies for advantages, a caseworker reviews the applying for irregularities. If an application looks suspicious, it might probably be sent to the town’s investigations department—which may lead to a rejection, a request to correct paperwork errors, or a suggestion that the candidate receive less money. Investigations also can occur later, once advantages have been dispersed; the end result may force recipients to pay back funds, and even push some into debt.
Officials have broad authority over each applicants and existing welfare recipients. They will request bank records, summon beneficiaries to city hall, and in some cases make unannounced visits to an individual’s home. As investigations are carried out—or paperwork errors fixed—much-needed payments could also be delayed. And sometimes—in greater than half of the investigations of applications, in keeping with figures provided by Bodaar—the town finds no evidence of wrongdoing. In those cases, this could mean that the town has “wrongly harassed people,” Bodaar says.
The Smart Check system was designed to avoid these scenarios by eventually replacing the initial caseworker who flags which cases to send to the investigations department. The algorithm would screen the applications to discover those probably to involve major errors, based on certain personal characteristics, and redirect those cases for further scrutiny by the enforcement team.
If all went well, the town wrote in its internal documentation, the system would improve on the performance of its human caseworkers, flagging welfare applicants for investigation while identifying a proportion of cases with errors. In a single document, the town projected that the model would prevent as much as 125 individual Amsterdammers from facing debt collection and save €2.4 million annually.
Smart Check was an exciting prospect for city officials like de Koning, who would manage the project when it was deployed. He was optimistic, for the reason that city was taking a scientific approach, he says; it might “see if it was going to work” as an alternative of taking the attitude that “this must work, and irrespective of what, we’ll proceed this.”
It was the type of daring concept that attracted optimistic techies like Loek Berkers, an information scientist who worked on Smart Check in just his second job out of school. Speaking in a restaurant tucked behind Amsterdam’s city hall, Berkers remembers being impressed at his first contact with the system: “Especially for a project inside the municipality,” he says, it “was very much a form of modern project that was trying something latest.”
Smart Check made use of an algorithm called an “explainable boosting machine,” which allows people to more easily understand how AI models produce their predictions. Most other machine-learning models are sometimes thought to be “black boxes” running abstract mathematical processes which are hard to grasp for each the staff tasked with using them and the people affected by the outcomes.
The Smart Check model would consider 15 characteristics—including whether applicants had previously applied for or received advantages, the sum of their assets, and the variety of addresses they’d on file—to assign a risk rating to everybody. It purposefully avoided demographic aspects, comparable to gender, nationality, or age, that were thought to steer to bias. It also tried to avoid “proxy” aspects—like postal codes—that won’t look sensitive on the surface but can develop into so if, for instance, a postal code is statistically related to a selected ethnic group.
In an unusual step, the town has disclosed this information and shared multiple versions of the Smart Check model with us, effectively inviting outside scrutiny into the system’s design and performance. With this data, we were capable of construct a hypothetical welfare recipient to get insight into how a person applicant can be evaluated by Smart Check.
This model was trained on an information set encompassing 3,400 previous investigations of welfare recipients. The thought was that it might use the outcomes from these investigations, carried out by city employees, to work out which aspects within the initial applications were correlated with potential fraud.
