modeling is the top of analytics value. It doesn’t give attention to what happened, and even what occur – it takes analytics further by telling us what we should always do to vary what occur. To harness this extra prescriptive power, nevertheless, we must tackle a further assumption…a causal assumption. The naive practitioner will not be aware that moving from predictive to prescriptive comes with the luggage of this lurking assumption. I Googled ‘prescriptive analytics’ and searched the primary ten articles for the word ‘causal.’ To not my surprise (but to my disappointment), I didn’t get a single hit. I loosened the specificity of my word search by trying ‘assumption’ – this one did surprise me, not a single hit either! It is obvious to me that that is an under-taught component of prescriptive modeling. Let’s fix that!
Whenever you use prescriptive modeling, you make causal bets, whether you understand it or not. And from what I’ve seen this can be a terribly under-emphasized point on the subject given its importance.
By the tip of this text, you should have a transparent understanding of why prescriptive modeling has causal assumptions and the way you may discover in case your model/approach meets them. We’ll get there by covering the topics below:
- Temporary overview of prescriptive modeling
- Why does prescriptive modeling have a causal assumption?
- How will we know if we have now met the causal assumption?
What’s Prescriptive Modeling?
Before we get too far, I would like to say that that is an article on prescriptive analytics – there’s plenty of knowledge about that elsewhere. This portion will probably be a fast overview to function a refresher for readers who’re already at the very least somewhat conversant in the subject.
There’s a widely known hierarchy of three analytics types: (1) descriptive analytics, (2) predictive analytics, and (3) prescriptive analytics.
Descriptive analytics looks at attributes and qualities in the info. It calculates trends, averages, medians, standard deviations, etc. Descriptive analytics doesn’t try to say anything more in regards to the data than is empirically observable. Often, descriptive analytics are present in dashboards and reports. The worth it provides is in informing the user of the important thing statistics in the info.
Predictive analytics goes a step beyond descriptive analytics. As an alternative of summarizing data, predictive analytics finds relationships inside the info. It attempts to separate the noise from the signal in these relationships to seek out underlying, generalizable patterns. From those patterns, it may possibly make predictions on unseen data. It goes further than descriptive analytics since it provides insights on unseen data, somewhat than simply the info which can be immediately observed.
Prescriptive analytics goes a further step beyond predictive analytics. Prescriptive analytics uses models created through predictive analytics to recommend smart or optimal actions. Often, prescriptive analytics will run simulations through predictive models and recommend the strategy with probably the most desirable consequence.
Let’s consider an example to raised illustrate the difference between predictive and prescriptive analytics. Imagine you’re an information scientist at an organization that sells subscriptions to online publications. You’ve developed a model that predicts that probability that a customer will cancel their subscription in a given month. The model has multiple inputs, including promotions sent to the client. To this point, you’ve only engaged in predictive modeling. At some point, you get the brilliant concept that it’s best to input different discounts into your predictive model, observe the impact of the discounts on customer churn, and recommend the discounts that best balance the associated fee of the discount with the good thing about increased customer retention. Along with your shift in focus from prediction to intervention, you could have graduated to prescriptive analytics!
Below are examples of possible analyses for the client churn model for every level of analytics:
Now that we’ve been refreshed on the three varieties of analytics, let’s get into the causal assumption that is exclusive to prescriptive analytics.
The Causal Assumption in Prescriptive Analytics
Moving from predictive to prescriptive analytics feels intuitive and natural. You’ve a model that predicts a vital consequence using features, a few of that are in your control. It is sensible to then simulate manipulating those features to drive towards a desired consequence. What doesn’t feel intuitive (at the very least to a junior modeler) is that doing so moves you right into a dangerous space in case your model hasn’t captured the causal relationships between the goal variable and the features you propose to vary.
We’ll first show the hazards with an easy example involving a rubber duck, leaves and a pool. We’ll then move on to real-world failures which have come from making causal bets after they weren’t warranted.
Leaves, a pool and a rubber duck
You enjoy spending time outside near your pool. As an astute observer of your environment, you notice that your favorite pool toy – a rubber duck – is often in the identical a part of the pool because the leaves that fall from a close-by tree.
Eventually, you choose that it’s time to clean the leaves out of the pool. There’s a particular corner of the pool that’s easiest to access, and you would like the entire leaves to be in that area so you may more easily collect and discard them. Given the model you could have created – the rubber duck is in the identical area because the leaves – you choose that it might be very clever to maneuver the toy to the corner and watch in delight because the leaves follow the duck. Then you definitely will easily scoop them up and proceed with the remaining of the day, having fun with your newly cleaned pool.
You make the change and feel like a idiot as you stand within the corner of the pool, right over the rubber duck, net in hand, while the leaves stubbornly stay in place. You’ve made the terrible mistake of using prescriptive analytics when your model doesn’t pass the causal assumption!
Perplexed, you look into the pool again. You notice a slight disturbance within the water coming from the pool jets. You then resolve to rethink your predictive modeling approach using the angle of the jets to predict the placement of the leaves as an alternative of the rubber duck. With this recent model, you estimate how you want to configure the jets to get the leaves to your favorite corner. You progress the jets and this time you’re successful! The leaves drift to the corner, you remove them and go on together with your day a wiser data scientist!
This can be a quirky example, but it surely does illustrate a couple of points well. Let me call them out.
- The rubber duck is a classic ‘confounding’ variable. Additionally it is affected by the pool jets and has no impact on the placement of the leaves.
- Each the rubber duck and the pool jet models made accurate predictions – if we simply desired to know where the leaves were, they could possibly be equivalently good.
- What breaks the rubber duck model has nothing to do with the model itself and every little thing to do with the way you the model. The causal assumption wasn’t warranted but you moved forward anyway!
I hope you enjoyed the whimsical example – let’s transition to talking about real-world examples.
Shark Tank Pitch
In case you haven’t seen it, Shark Tank is a show where entrepreneurs pitch their business idea to wealthy investors (called ‘sharks’) with the hopes of securing investment money.
I used to be recently watching a Shark Tank re-run (as one does) – one in every of the pitches within the episode (Season 10, Episode 15) was for an organization called GoalSetter. GoalSetter is an organization that permits parents to open ‘mini’ bank accounts of their child’s name that family and friends could make deposits into. The thought is that as an alternative of giving toys or gift cards to children as presents, people may give deposit certificates and kids can save up for things (‘goals’) they need to purchase.
I actually have no qualms with the business idea, but within the presentation, the entrepreneur made this claim:
…kids who’ve savings accounts of their name are six times more more likely to go to school and 4 times more more likely to own stocks by the point they’re young adults…
Assuming this statistic is true, this statement, by itself, is all advantageous and well. We will have a look at the info and see that there’s a relationship between a toddler having a checking account of their name and going to school and/or investing (descriptive). We could even develop a model that predicts if a toddler will go to school or own stocks using checking account of their name as a predictor (predictive). But this doesn’t tell us anything about causation! The investment pitch has this subtle prescriptive message – “give your kid a GoalSetting account they usually will probably be more more likely to go to school and own stocks.” While semantically just like the quote above, these two statements are worlds apart! One is an announcement of statistical incontrovertible fact that relies on no assumptions, and the opposite is a prescriptive statement that has a huge causal assumption! I hope that confounding variable alarms are ringing in your head immediately. It seems much likely that things like household income, financial literacy of fogeys and cultural influences would have a relationship with each the probability of opening a checking account in a toddler’s name and that child going to school. It doesn’t seem likely that giving a random kid a checking account of their name will increase their probabilities of going to school. That is like moving the duck within the pool and expecting the leaves to follow!
Reading Is Fundamental Program
Within the Nineteen Sixties, there was a government-funded program called ‘Reading is Fundamental (RIF).’ A part of this program focused on putting books within the homes of low-income children. The goal was to extend literacy in those households. The strategy was partially based on the concept homes with more books in them had more literate children. You would possibly know where I’m going with this one based on the Shark Tank example we just discussed. Observing that homes with a number of books have more literate children is descriptive. There’s nothing unsuitable with that. But, once you start making recommendations, you step out of descriptive space and leap into the prescriptive world – and as we’ve established, that comes with the causal assumption. Putting books in homes assumes that the books cause the literacy! Research by Susan Neuman found that putting books in homes was not sufficient in increasing literacy without additional resources1.
After all, giving books to children who can’t afford them is an excellent thing – you don’t need a causal assumption to do good things 😊. But, if you could have the particular goal of accelerating literacy, you can be well-advised to evaluate the validity of the causal assumption behind your actions to understand your required results!
How will we know if we satisfy the causality assumption?
We’ve established that prescriptive modeling requires a causal assumption (a lot that you simply are probably exhausted!). But how can we all know if the idea is met by our model? When fascinated about causality and data, I find it helpful to separate my thoughts between experimental and observational data. Let’s undergo how we will feel good (or possibly at the very least ‘okay’) about causal assumptions with these two varieties of data.
Experimental Data
If you could have access to experimental data to your prescriptive modeling, you’re very lucky! Experimental data is the gold standard for establishing causal relationships. The small print of why that is the case are out of scope of this text, but I’ll say that the randomized task of treatments in a well-designed experiment deals with confounders, so that you don’t should worry about them ruining your casual assumptions.
We will train predictive models on the output of an excellent experiment – i.e., good experimental data. On this case, the data-generating process meets causal identification conditions between the goal variables and variables that were randomly assigned treatments. I would like to emphasise that only variables which can be randomly assigned within the experiment will qualify for the causal claim on the premise of the experiment alone. The causal effect of other variables (called covariates) may or will not be appropriately captured. For instance, imagine that we ran an experiment that randomly provided multiple plants with various levels of nitrogen, phosphorus and potassium and we measured the plant growth. From this experimental data, we created the model below:
Because nitrogen, phosphorus and potassium were treatments that were randomly assigned within the experiment, we will conclude that betas 1 through 3 estimate a causal relationship on plant growth. Sun exposure was not randomly assigned which prevents us from claiming a causal relationship through the ability of experimental data. This is just not to say that a causal claim will not be justified for covariates, however the claim would require additional assumptions that we are going to cover within the observational data section coming up.
I’ve used the qualifier when talking about experimental data multiple times now. What’s a experiment? I’ll go over two common issues I’ve seen that prevent an experiment from creating good data, but there’s loads more that may go unsuitable. You need to read up on experimental design if you happen to would really like to go deeper.
: That is one of the common issues with experiments. I used to be once assigned to a project a couple of years ago where an experiment was run, but some data were mixed up regarding which subjects got which treatments – the info was not usable! If there have been significant execution mistakes chances are you’ll not have the ability to attract valid causal conclusions from the experimental data.
: This could occur for multiple reasons – for instance, there will not be enough signal coming from the treatment, or there could have been too few experimental units. Even with perfect execution, an underpowered study may fail to uncover real effects which could prevent you from meeting the causal conclusion required for prescriptive modeling.
Observational Data
Satisfying the causal assumption with observational data is rather more difficult, dangerous and controversial than with experimental data. The randomization that could be a key part in creating experimental data is powerful since it removes the issues brought on by all confounding variables – known and unknown, observed and unobserved. With observational data, we don’t have access to this extremely useful power.
Theoretically, if we will control for confounding variables, we will still make causal claims with observational data. While some may disagree with this statement, it’s widely accepted in principle. The actual challenge lies in the applying.
To appropriately control for a confounding variable, we want to (1) have high-quality data for the variable and (2) appropriately model the connection between the confounder and our goal variable. Doing this for every known confounder is difficult, but it surely isn’t the worst part. The worst part is you could never know with certainty that you could have accounted for all confounders. Even with strong domain knowledge, the chance that there’s an unknown confounder “on the market” stays. The most effective we will do is include every confounder we will consider after which depend on what known as the ‘no unmeasured confounder’ assumption to estimate causal relationships.
Modeling with observational data can still add a variety of value in prescriptive analytics, although we will never know with certainty that we accounted for all confounding variables. With observational data, I believe of the causal assumption as being met in degrees as an alternative of in a binary fashion. As we account for more confounders, we capture the causal effect higher and higher. Even when we miss a couple of confounders, the model should add value. So long as the confounders don’t have too large of an impact on the estimated causal relationships, we may have the ability so as to add more value making decisions with a rather biased causal model than using the method we had before we used prescriptive modeling (e.g., rules or intuition-based decisions).
Having a practical mindset with observational data will be necessary since (1) observational data is cheaper and rather more common than experimental data and (2) if we depend on airtight causal conclusions (which we will’t get with observational data), we could also be leaving value on the table by ruling out causal models which can be ‘adequate’, though not perfect. You and your small business partners have to choose the extent of leniency to have with meeting the causal assumption, a model built on observational data could still add major value!
Wrapping it up
While prescriptive analytics is powerful and has the potential so as to add a variety of value, it relies on causal assumptions while descriptive and predictive analytics don’t. It’s important to grasp and to satisfy the causal assumption in addition to possible.
Experimental data is the gold standard of estimating causal relationships. A model built on good experimental data is in a robust position to satisfy the causal assumptions required by prescriptive modeling.
Establishing causal relationships with observational data will be harder due to the potential of unknown or unobserved confounding variables. We must always balance rigor and pragmatism when using observational data for prescriptive modeling – rigor to consider and attempt to regulate for each confounder possible and pragmatism to grasp that while the causal effects will not be perfectly captured, the model may add more value than the present decision-making process.
I hope that this text has helped you gain a greater understanding of why prescriptive modeling relies on causal assumptions and the way you may address meeting those assumptions. Completely happy modeling!
- Neuman, S. B. (2017). . Teachers College Record, 119(6), 1–32.
