In keeping with Gartner in 2019, “85% of AI Implementations Will Fail By 2022”. Whether this dire prediction proved to be true is irrelevant. I hereby propose a possible explanation for why AI implementations may fail.
When using Machine Learning models or computing statistics, our ultimate goal is to base our future decisions on the expected outcomes.
For example, when making a model to detect potential churners, the aim is to take motion to retain them.
One other example: Market Basket Evaluation is normally used to achieve insight into incessantly purchased products so as to inform product recommendations.
My intuition is that this approach results in two most important problems:
The primary issue with taking motion based on Machine Learning is that the very motion undertaken in consequence of the model’s predictions changes the information used to create the model, making the model’s predictions inaccurate or biased.
When using a model to predict client churn, the results of retaining potential churners results in a change within the behaviour of those clients, resulting in a difference between the brand new data set and the previous data set used to create the model. That is akin to the infamously phenomenon called “Data Drift”. Only this time, the drift is created by way of the model itself.
Likewise, when recommending products based on incessantly purchased items, this leads to artificially increasing sales for those products, which then further perpetuates the recommendations. This creates a self-reinforcing “filter bubble”.
One other issue arises after we use Machine Learning to derive feature importances and derive actions: in doing so, we frequently mistake correlations for actual causes.
Consider two distinct philosophical perspectives: a passive observer associates the crowing of a rooster with sunrise, but cannot understand its cause. To him, the rooster’s crow is solely an indication that the sun is rising. The action-based observer, alternatively, realizes that forcing the rooster to crow is not going to cause the sun to rise. To him, correlations should not enough.
Today, too many Data Scientists and Marketing Practitioners behave as if the proverbial crowing causes the sun to rise. Unfortunately, it isn’t as easy as mining past marketing campaigns’ data, using AI to model a predicting task, identifying feature importances and leveraging them to tell our future decisions. In doing so, they are sometimes mistaking correlations with causes.
Likewise, to cite a famous example, for a passive observer, beers and diapers are correlated. For the action-based observer, putting beers and diapers next to one another may create an exposure bias and reduce total revenues.
As we are able to see, these are two very different perspectives. Most Machine Learning models adopt a Passive Observer standpoint, yet we use them as in the event that they were learned with the Motion-based Observer perspective.
This looks as if an unattainable equation to unravel for AI. As soon as an AI model is trained, it tends to change into quickly unusable. As Hegel put it, “The owl of Minerva spreads its wings only with dusk.” (by which he meant that we are able to only understand things in hindsight).
This series of posts
Within the upcoming articles, we’ll delve into the taxonomy of biases that arise from the Passive Observer standpoint and explore the important thing methods for adopting the Motion-Based Observer standpoint in the sector of Machine Learning.