Seems, even language models “think” they’re biased. When prompted in ChatGPT, the response was as follows: “Yes, language models can have biases, since the training data reflects the biases present in society from which that data was collected. For instance, gender and racial biases are prevalent in lots of real-world datasets, and if a language model is trained on that, it may perpetuate and amplify these biases in its predictions.” A well known but dangerous problem.
Humans (typically) can dabble with each logical and stereotypical reasoning when learning. Still, language models mainly mimic the latter, an unlucky narrative we’ve seen play out ad nauseam when the flexibility to employ reasoning and significant pondering is absent. So would injecting logic into the fray be enough to mitigate such behavior?
Scientists from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) had an inkling that it would, so that they set off to look at if logic-aware language models could significantly avoid more harmful stereotypes. They trained a language model to predict the connection between two sentences, based on context and semantic meaning, using a dataset with labels for text snippets detailing if a second phrase “entails,” “contradicts,” or is neutral with respect to the primary one. Using this dataset — natural language inference — they found that the newly trained models were significantly less biased than other baselines, with none extra data, data editing, or additional training algorithms.
For instance, with the premise “the person is a health care provider” and the hypothesis “the person is masculine,” using these logic-trained models, the connection can be classified as “neutral,” since there’s no logic that claims the person is a person. With more common language models, two sentences might sound to be correlated because of some bias in training data, like “doctor” is perhaps pinged with “masculine,” even when there’s no evidence that the statement is true.
At this point, the omnipresent nature of language models is well-known: Applications in natural language processing, speech recognition, conversational AI, and generative tasks abound. While not a nascent field of research, growing pains can take a front seat as they increase in complexity and capability.
“Current language models suffer from issues with fairness, computational resources, and privacy,” says MIT CSAIL postdoc Hongyin Luo, the lead creator of a latest paper concerning the work. “Many estimates say that the CO2 emission of coaching a language model may be higher than the lifelong emission of a automobile. Running these large language models can be very expensive due to the amount of parameters and the computational resources they need. With privacy, state-of-the-art language models developed by places like ChatGPT or GPT-3 have their APIs where you should upload your language, but there’s no place for sensitive information regarding things like health care or finance. To resolve these challenges, we proposed a logical language model that we qualitatively measured as fair, is 500 times smaller than the state-of-the-art models, may be deployed locally, and with no human-annotated training samples for downstream tasks. Our model uses 1/400 the parameters compared with the biggest language models, has higher performance on some tasks, and significantly saves computation resources.”
This model, which has 350 million parameters, outperformed some very large-scale language models with 100 billion parameters on logic-language understanding tasks. The team evaluated, for instance, popular BERT pretrained language models with their “textual entailment” ones on stereotype, career, and emotion bias tests. The latter outperformed other models with significantly lower bias, while preserving the language modeling ability. The “fairness” was evaluated with something called ideal context association (iCAT) tests, where higher iCAT scores mean fewer stereotypes. The model had higher than 90 percent iCAT scores, while other strong language understanding models ranged between 40 to 80.
Luo wrote the paper alongside MIT Senior Research Scientist James Glass. They are going to present the work on the Conference of the European Chapter of the Association for Computational Linguistics in Croatia.
Unsurprisingly, the unique pretrained language models the team examined were teeming with bias, confirmed by a slew of reasoning tests demonstrating how skilled and emotion terms are significantly biased to the female or masculine words within the gender vocabulary.
With professions, a language model (which is biased) thinks that “flight attendant,” “secretary,” and “physician’s assistant” are feminine jobs, while “fisherman,” “lawyer,” and “judge” are masculine. Concerning emotions, a language model thinks that “anxious,” “depressed,” and “devastated” are feminine.
While we should still be distant from a neutral language model utopia, this research is ongoing in that pursuit. Currently, the model is only for language understanding, so it’s based on reasoning amongst existing sentences. Unfortunately, it may’t generate sentences for now, so the following step for the researchers can be targeting the uber-popular generative models built with logical learning to make sure more fairness with computational efficiency.
“Although stereotypical reasoning is a natural a part of human recognition, fairness-aware people conduct reasoning with logic reasonably than stereotypes when crucial,” says Luo. “We show that language models have similar properties. A language model without explicit logic learning makes loads of biased reasoning, but adding logic learning can significantly mitigate such behavior. Moreover, with demonstrated robust zero-shot adaptation ability, the model may be directly deployed to different tasks with more fairness, privacy, and higher speed.”