Nora Petrova, is a Machine Learning Engineer & AI Consultant at Prolific. Prolific was founded in 2014 and already counts organizations like Google, Stanford University, the University of Oxford, King’s College London and the European Commission amongst its customers, using its network of participants to check latest products, train AI systems in areas like eye tracking and determine whether their human-facing AI applications are working as their creators intended them to.
Could you share some information in your background at Prolific and profession thus far? What got you fascinated by AI?
My role at Prolific is split between being an advisor regarding AI use cases and opportunities, and being a more hands-on ML Engineer. I began my profession in Software Engineering and have step by step transitioned to Machine Learning. I’ve spent many of the last 5 years focused on NLP use cases and problems.
What got me fascinated by AI initially was the flexibility to learn from data and the link to how we, as humans, learn and the way our brains are structured. I feel ML and Neuroscience can complement one another and help further our understanding of how one can construct AI systems which are able to navigating the world, exhibiting creativity and adding value to society.
What are among the biggest AI bias issues that you just are personally aware of?
Bias is inherent in the information we feed into AI models and removing it completely could be very difficult. Nevertheless, it’s imperative that we’re aware of the biases which are in the information and find ways to mitigate the harmful sorts of biases before we entrust models with essential tasks in society. The most important problems we’re facing are models perpetuating harmful stereotypes, systemic prejudices and injustices in society. We must always be mindful of how these AI models are going for use and the influence they are going to have on their users, and be sure that they’re protected before approving them for sensitive use cases.
Some outstanding areas where AI models have exhibited harmful biases include, the discrimination of underrepresented groups in class and university admissions and gender stereotypes negatively affecting recruitment of ladies. Not only this however the a criminal justice algorithm was found to have mislabeled African-American defendants as “high risk” at nearly twice the speed it mislabeled white defendants within the US, while facial recognition technology still suffers from high error rates for minorities as a result of lack of representative training data.
The examples above cover a small subsection of biases demonstrated by AI models and we are able to foresee greater problems emerging in the longer term if we don’t concentrate on mitigating bias now. It is crucial to remember that AI models learn from data that contain these biases as a result of human decision making influenced by unchecked and unconscious biases. In a number of cases, deferring to a human decision maker may not eliminate the bias. Truly mitigating biases will involve understanding how they’re present in the information we use to coach models, isolating the aspects that contribute to biased predictions, and collectively deciding what we would like to base essential decisions on. Developing a set of standards, in order that we are able to evaluate models for safety before they’re used for sensitive use cases can be a very important step forward.
AI hallucinations are an enormous problem with any kind of generative AI. Are you able to discuss how human-in-the-loop (HITL) training is capable of mitigate these issues?
Hallucinations in AI models are problematic specifically use cases of generative AI but it is vital to notice that they aren’t an issue in and of themselves. In certain creative uses of generative AI, hallucinations are welcome and contribute towards a more creative and interesting response.
They might be problematic in use cases where reliance on factual information is high. For instance, in healthcare, where robust decision making is vital, providing healthcare professionals with reliable factual information is imperative.
HITL refers to systems that allow humans to supply direct feedback to a model for predictions which are below a certain level of confidence. Inside the context of hallucinations, HITL might be used to assist models learn the extent of certainty they need to have for various use cases before outputting a response. These thresholds will vary depending on the use case and teaching models the differences in rigor needed for answering questions from different use cases can be a key step towards mitigating the problematic sorts of hallucinations. For instance, inside a legal use case, humans can exhibit to AI models that fact checking is a required step when answering questions based on complex legal documents with many clauses and conditions.
How do AI staff akin to data annotators help to cut back potential bias issues?
AI staff can at the beginning help with identifying biases present in the information. Once the bias has been identified, it becomes easier to give you mitigation strategies. Data annotators also can help with coming up with ways to cut back bias. For instance, for NLP tasks, they will help by providing other ways of phrasing problematic snippets of text such that the bias present within the language is reduced. Moreover, diversity in AI staff will help mitigate issues with bias in labelling.
How do you be sure that the AI staff aren’t unintentionally feeding their very own human biases into the AI system?
It’s definitely a fancy issue that requires careful consideration. Eliminating human biases is sort of not possible and AI staff may unintentionally feed their biases to the AI models, so it is vital to develop processes that guide staff towards best practices.
Some steps that might be taken to maintain human biases to a minimum include:
- Comprehensive training of AI staff on unconscious biases and providing them with tools on how one can discover and manage their very own biases during labelling.
- Checklists that remind AI staff to confirm their very own responses before submitting them.
- Running an assessment that checks the extent of understanding that AI staff have, where they’re shown examples of responses across several types of biases, and are asked to decide on the least biased response.
Regulators internationally are meaning to regulate AI output, what in your view do regulators misunderstand, and what have they got right?
It is crucial to begin by saying that this can be a really difficult problem that no person has found the answer to. Society and AI will each evolve and influence each other in ways which are very difficult to anticipate. An element of an efficient strategy for locating robust and useful regulatory practices is being attentive to what is occurring in AI, how persons are responding to it and what effects it has on different industries.
I feel a big obstacle to effective regulation of AI is a lack of know-how of what AI models can and can’t do, and the way they work. This, in turn, makes it harder to accurately predict the implications these models could have on different sectors and cross sections of society. One other area that’s lacking is assumed leadership on how one can align AI models to human values and what safety looks like in additional concrete terms.
Regulators have sought collaboration with experts within the AI field, have been careful to not stifle innovation with overly stringent rules around AI, and have began considering consequences of AI on jobs displacement, that are all very essential areas of focus. It is crucial to string rigorously as our thoughts on AI regulation make clear over time and to involve as many individuals as possible as a way to approach this issue in a democratic way.
How can Prolific solutions assist enterprises with reducing AI bias, and the opposite issues that we’ve discussed?
Data collection for AI projects hasn’t all the time been a considered or deliberative process. We’ve previously seen scraping, offshoring and other methods running rife. Nevertheless, how we train AI is crucial and next-generation models are going to must be built on intentionally gathered, prime quality data, from real people and from those you’ve gotten direct contact with. That is where Prolific is making a mark.
Other domains, akin to polling, market research or scientific research learnt this a protracted time ago. The audience you sample from has a huge impact on the outcomes you get. AI is starting to catch up, and we’re reaching a crossroads now.
Now’s the time to begin caring about using higher samples begin and dealing with more representative groups for AI training and refinement. Each are critical to developing protected, unbiased, and aligned models.
Prolific will help provide the correct tools for enterprises to conduct AI experiments in a protected way and to gather data from participants where bias is checked and mitigated along the way in which. We will help provide guidance on best practices around data collection, and selection, compensation and fair treatment of participants.
What are your views on AI transparency, should users have the ability to see what data an AI algorithm is trained on?
I feel there are pros and cons to transparency and an excellent balance has not yet been found. Corporations are withholding information regarding data they’ve used to coach their AI models as a result of fear of litigation. Others have worked towards making their AI models publicly available and have released all information regarding the information they’ve used. Full transparency opens up a number of opportunities for exploitation of the vulnerabilities of those models. Full secrecy doesn’t help with constructing trust and involving society in constructing protected AI. An excellent middle ground would offer enough transparency to instill trust in us that AI models have been trained on good quality relevant data that we now have consented to. We want to pay close attention to how AI is affecting different industries and open dialogues with affected parties and be certain that we develop practices that work for everybody.
I feel it’s also essential to contemplate what users would find satisfactory by way of explainability. In the event that they want to grasp why a model is producing a certain response, giving them the raw data the model was trained on probably won’t help with answering their query. Thus, constructing good explainability and interpretability tools is significant.
AI alignment research goals to steer AI systems towards humans’ intended goals, preferences, or ethical principles. Are you able to discuss how AI staff are trained and the way that is used to make sure the AI is aligned as best as possible?
That is an lively area of research and there isn’t consensus yet on what strategies we should always use to align AI models to human values and even which set of values we should always aim to align them to.
AI staff are frequently asked to authentically represent their preferences and answer questions regarding their preferences truthfully whilst also adhering to principles around safety, lack of bias, harmlessness and helpfulness.
Regarding alignment towards goals, ethical principles or values, there are multiple approaches that look promising. One notable example is the work by The Meaning Alignment Institute on Democratic Advantageous-Tuning. There is a wonderful post introducing the concept here.
