Why we must always move to a world where analyses are the default first step

It’s been nearly 2 years since Seth Rosen’s trashboard tweet, and yet the trashboard epidemic continues to be quite real. We still encounter firms which are curiously completely satisfied with having 1000’s of dashboards, their analysts left to spelunk deep caverns of ad hoc requests with naught but an IDE.

There are good intentions throughout, in fact — ad hoc work might be painful, and an automatic solution sounds appealing. What’s more, dashboarding is a secure bet — we still live in a world where the dashboard continues to be accepted as The Default Tool For Analytics, in spite of everything.

Let’s speak about how we came, why this needs to vary, and where we must always go from here. Briefly, in what follows I’ll argue that:

• A long time of heritage have brought us to a weird place where dashboards have primacy.
• But dashboards should not all the time the most effective method to provide value to your org, especially after the primary few.
• When you imagine the 2 statements above, your beliefs, your processes, your tooling have to fundamentally change.

Dashboards vs. analyses, by means of cooking

So don’t misunderstand — dashboards are implausible, they’re just blunt. They will serve limitless needs, but in doing so, rarely do they serve particular needs perfectly. It’s the Heisenberg uncertainty principle for data: the more data you expose, the less sharp your insight. Dashboards, by nature, prioritize data exposure. They provide data by itself, and interpretation is left to the patron. A certain variety of core dashboards are actually required, in fact. Every business unit has core metrics and events that it tracks, and these need to be surfaced somewhere — the dashboard here acts because the heartbeat of the business. But beyond this, value greatly diminishes after each nth dashboard.

Analyses, however, are sharp. They expose interpretation, enabling more precise relevance to business problems. If dashboards are like salad bars, analyses are made-to-order meals. And just as specific meals fit cravings higher, so too will analyses all the time fit specific business needs higher. Then again, incremental salad bar items (more dashboards) have diminishing returns — they’ll never quite satisfy a specific craving, but they’ll crudely solve most hunger-related problems. You possibly can profit greatly from having a number of, but you don’t have to proceed constructing them endlessly.

Then why do dashboards still have primacy?

Although I’ve found folks generally agree on that sentiment, we live in a peculiar time in history where dashboarding still retains its primacy. It made sense when data was slow, data was hard to handle, and dashboards were cumbersome to make — core dashboard maintenance was everything of an analyst’s job. But because the warehouse sped up, as analytics needs modified, because the returns on additional dashboards diminished, our mindset never adjusted accordingly.

We’ve in some way turn into trapped by some perversion of the sunk cost fallacy: we’d allocated all incremental resources to dashboarding, and so we began to imagine dashboards were of the utmost importance.

I’ll concede that there’s an inherent attractiveness to dashboards: dashboards allow us to construct with high coverage in mind. They sell the dream that our ad hoc request volume will drop to 0 someday. At some point, given enough dashboards, we’ll have addressed all questions and all follow-up questions which are to return.

But unfortunately, running a business isn’t so formulaic. Power law distributions proliferate. Preferential attachment begets pockets of value and worthlessness, change and stagnation, accumulation and void. Not all things are equal. It’s the world of the Pareto principle, where avalanches drive change. Where 20% of the trouble drives 80% of the outcomes.

And on this world, analyses are not any exception. When you were to rank order all of the work you’ve done by impact, I believe you’ll find most of your value add got here from just a number of key analyses. And on this world, the dashboard flounders — what we gain in exhaustiveness, we rob from the avalanche.

I’ve long been a proponent of elevating the role of the analyst. But I seldom do analytics work nowadays, and after I recently got my first ad hoc request in months, I’ll admit I jumped directly into the info. I had a robust urge to reply the query as quickly as possible — I viewed the work as a hindrance, not a chance. It was an try and diminish the work required of me, but in doing so, I fell from Co-founder to human API. I fell victim to the bad loop ruining analytics, at the same time as a self-proclaimed curate of this borderline analytics religion.

“Every thing is a self portrait. Every thing is a diary. If you care about something it becomes fairly obvious to everyone else. If you care, your devotion or discipline is directional. Most individuals can intuitively feel it.”
– Chuck Palahniuk

Ergo: we want to vary our defaults — our beliefs, our processes, our tools — or risk falling back into the usual habits.

Belief: value beyond data pulls.

Step one in driving any type of behavior change is to merely internalize that there’s a transparent reason why you’re doing what you’re doing. We’d like to imagine that our worth add extends beyond getting data. By all means, should you imagine that your organization’s primary need is a core set of dashboards, then set those up and focus your efforts on that. But when you will have an inkling you can provide leverage outside of this — by helping folks make higher decisions, or by even keeping others intellectually honest within the conversions — then internalize this and proceed under the belief that it’s true.

Processes: analysis-by-default, not dashboard-by-default.

To some extent, I feel all of us wish to have more impactful roles, drive more business value, take part in higher levels of strategic problem-solving. But beyond simply recognizing that, the remaining hurdles are still substantial. In my estimation, essentially the most debilitating hindrance henceforth is that it’s far too easy to fall into bad habits. We’d like to construct systems that make it easy to do the best things and harder to do the mistaken ones.

“Goals are good for setting a direction, but systems are best for making progress.”
– James Clear, Atomic Habits

I’d start by considering a move to an analysis-by-default world, particularly should you’ve already invested heavily into self-service systems and a set of core dashboards. If we spend our IC time attempting to maximize the usefulness of our work (evaluation), slightly than attempting to maximize the scalability of our work (dashboards), it follows then that our work will find yourself being more useful, by definition.

Beyond that, construct templates, standards around how work needs to be done. Write up hardened operating principles to your team. You may even set rigid policies to implement higher behavior: analyses don’t count unless you set them in our centralized, agreed-upon place, as an example. Adopt rituals that reinforce good behavior: have each day stand-ups where you present this work; bake ritual compliance into your performance reviews. Do whatever it takes to ensure that folks do the things that you realize will make them higher.

If we imagine in analysis-by-default, we want a set of tools that inverts common working patterns and pushes us to construct analyses first, then dashboards, not vice versa. Actually I’m biased — this has been the philosophy behind constructing Hyperquery — but even should you’re not sold on our vision, even ensuring you don’t jump into Tableau for each evaluation is a worthy first step. At minimum, narrative-first is a must. By simply forcing yourself to default to words as deliverables, slightly than data, you’re already breaking out of a ruinous habit.

Once I was at Airbnb, I’d often get questions from my stakeholders. And for a very long time, my first instinct was to leap to SQL, immediately. We’ve all done this: in the will to offer quick value, we go full Minority Report, live coding. There’s a time and place for this, but we’re far too wanting to go there when we must always pondering as a substitute — synthesizing and drawing on learnings from the wealth of past experiments and deep dives we’ve already conducted. We fail to offer leveraged value, and as a substitute seek approval for our flashy (unrivaled, sure!) technical prowess. And this quietly becomes the worth others expect of us.

Narrative-first forces alignment, relevance, value over immediacy. Narrative-first elevates the conversation beyond technical contributions to strategic contributions. Narrative-first nudges to your stakeholders that your value lies in narratives, not code.

Analytics is ripe for change. Even should you don’t buy the arguments I’ve laid out, it’s undeniable that our industry has modified astronomically within the last decade. And with recent cars come recent roads, recent laws. At minimum, I hope this post pushes you to take into consideration what that recent world ought to appear to be.