|
This edition of TMAI Premium is free for Standard Edition subscribers.Â
Over the last few weeks, TMAI Premium editions have covered mind-stretching topics. Six Big Little Economic Ideas was a two-part series on how to tackle difficult (daily) challenges we face via non-usual influence. I had so much fun compressing a lifetime of wisdom into TMAI 330, The Path to Measurement Nirvana. Last week, a handy guide on how to detect when others are manipulating your data/analysis! You can upgrade to TMAI Premium here - it comes with a money-back guarantee. All Premium revenue is donated to charity. | | TMAI #332: The Impact Matrix | Transforming Data’s Influence.
| I was grappling with a cluster of challenges: * How to assess how sophisticated a team's analytics practice is?
* What’s the best way to get leaders/Analysts away from low-value metrics? * Everyone wants a clear path to analytics glory, how to create one? * How to bring to fore the role of Machine Learnings & Automation? * What should be on the CMO dashboard, vs. a Director's? To answer these - very different - questions, I challenged myself to create something super simple, yet flexible. Something that could provide an effective assessment of reality, that was easy to execute. My stretch goal was to create a framework that might be instrumental in a Leader creating an inspiring vision for the future that was specific, focused, and actionable. The end result was a framework I call the impact matrix. Reflecting on it now, I’m amazed at what a wonderful tool continues to be - our company trains our best clients in using it to increase data's influence. It is my go to when I want to identify gaps in leadership thinking, and as a benchmark to judge progress/impact delivered via data. Let's walk through it together today. I'll be life changing, I promise. :) To assist your diagnostic purposes, there is a link to an XLS version at the end.[Programming Note: Like you, we’ve been using Conversion Lift Studies to measure channel-silo incrementality. Individual results are easy to report, but how do you create an incrementality dashboard that allows you to A. see the full picture clearly, and B. compare different campaigns/departments apples-to-apples? Next week’s Premium newsletter!] | The Foundational Structure.
My hope with the foundation was to create a structure that would be easily understood by deeply experienced Analysts AND extremely senior leaders who might have less familiarity with analytics details.
On the x-axis, how long does it take for a piece of data to become useful?Â
Not the timeliness, or frequency, at which data is collected. With every passing day, we can collect more and more data in real-time (and process it, store it, query the heck out of it). But, not all data becomes useful in real-time.
Some data becomes useful in real-time (a simple, humble metric like Impressions). Other data takes a while to accumulate enough intelligence to become useful (a complex, not-humble metric like Lifetime Value). |
|
On the y-axis, the size of decision the metric can impact.Â
The scale (see light gray) goes up dramatically as you rise along the y-axis.
If you read, learn, strategize and optimize Impressions, that decision is worth pennies.
If you read, learn, strategize and optimize for Lifetime Vale, that decision is worth millions of dollars (if not tens of millions).Â
Now that you have a grounding with those two anchors, you can lay out all the other metrics, and the metrics we honor with the moniker KPIs, in the matrix along the when useful and size of impact continuum. |
|
I’ve laid out the metrics we commonly use in online and offline marketing, metrics you’ll commonly see in digital analytics tool like Google Analytics, and your retail analytics reports as well. To ensure full coverage, I’ve included brand metrics as well as performance metrics.
You’ll also notice my obsession with pushing you, and our industry, on financial metrics above. Checkout where Gross Profit is, and where Incremental Profit is, etc.
You’ll also see easy metrics that are ok useful (Cost Per Acquisition) along with metrics that are omg this is freaking hard to measure (Profit Per Human) or (Profit Per Customer).Â
Note 1: The list above is not meant to be exhaustive. If you feel a super important metric is missing, find the optimal place for it (after you read the section below). Then, please, email me to let me know which metric you added to your impact matrix, and which position it occupies. (Excel version of mine is at the bottom of this email.)
Note 2: You can do the above exercise for any type of business analysis. Sales. Human Resources. Corporate Finance. Logistics. Whatever else. The principles, that different metrics mature to usefulness at different duration and each metric has a different sized impact, apply across they all.
Note 3: I’ve put a lot of thought into where each metric is. But, perhaps you disagree that Purchase Intent is two boxes lower than Checkout Abandonment Rate (two metrics that are so, so, difficult to compare). It is ok. If I’ve made you think, then I’ve succeeded. If you want to flip Purchase Intent’s position with Checkout Abandonment Rate because you’ve noticed something super special and super unique about your company… It is ok to flip them. Just, give it a deep think. | The Personal “where are we” Reflection.
Time to look into the mirror. (Boo!)
After I’d invented this way of laying out the metrics world, the first thing I wanted to use it to diagnose how sophisticated our team's practice of analytics was.
I did something simple: I color coded in green the metrics that were in the dashboards we were producing for the CxOs, and our highest profile Marketing reports.
Here’s the reflection in the mirror… | This team is working hard, they are measuring a lot of interesting metrics. Ex: % of Incremental Sales per Channel, that is actually a pretty hard thing to figure out.
The next step is to assess how sophisticated the analytics practice is, and what should the team focus on in terms of future priorities?
One of the cool things about having a matrix is that you can easily build a 2x2 - an approach that is excellent at delivering sharp focus. Besides, who does not want to be in the top right box? :)
Here's how I've matrixed  the impact matrix... |
|
[Originally, I’d called the bottom right quadrant “Basic.” Because that is what it is. But we are in marketing, we can dress it up. So “Basic” became “Solid foundation.” :)] Solid foundation has the cluster of metrics that are available easily (almost all standard metrics, pretty much all available for free from various tools), and they have low to better than low business impact. They are great for diagnostic purposes, but almost none of them will ever rise to earn that most royal of monikers: KPI. The Intermediate clusters are both high value. The one on the top left, very high business impact, worth loving lots. The one on the bottom right, takes time to become useful and needs lots of effort to achieve. The top right cluster is the hardest. It is stuffed with metrics that are almost always designated as KPIs, they take time to become useful (even when available in real-time), and any decision these KPIs power will be worth massive sums of revenue/profit. If you peek through the red shades above… You’ll notice that almost all the metrics in our dashboards were from the Solid Foundation cluster. Good accomplishment, but definitely not where we want to stop. Time to roll up our sleeves and get to actual work that will matter. And, we did. | Metrics = Employee Incentives. [Why Be Top Right.]
The reason I’m obsessed with the metrics chosen to be our KPIs is that metrics are essentially incentives you are creating for your employees. You pick the wrong one… Best case you will suck, normal case you will be hosed.
Here’s a good illustration, from a TMAI Premium edition earlier this year.Â
If you declare Users as your KPI, you will get loads and loads of traffic dumped on your site - relevant or now. Because, that is the incentive you’ve created.
As you move from left to right, you can imagine how the core incentives that will influence the behavior of your employees will change. |
|
Also, something obvious but hidden: If you pick Users, a Solid Foundation metric, as your KPI, you'll decide Organic Search is the best.
But, as you move to the right, the answer to which channel is best changes at every single step - as you move from Solid Foundation to Intermediate to Advanced.
You pick Profit as your KPI, an Advanced metric, the incentives for your employees about where to invest, what's crucial to solve for change completely.
Be super careful about the KPI you pick - it changes human behavior!
[TMAI Premium members, also see editions #313 & #314, Pick the Right Success KPI - P1 &  P2. If you can’t find them, ping me.] | The Incredibly Useful Impact Matrix "Slices."
I’d built the Impact Matrix on a 16-hour flight to Singapore. I was pretty pleased with getting as far as you see above.
But, there were another three hours to landing. So. I kept going. :)
One of the most difficult challenges we face is: How can I shift from puking data, and ensure the right individual is getting the right data for them?​
My approach was to leverage the y-axis to answer the above question, since what each layer of the org should receive is to be dictated by size of expected impact.Â
The solution was simple: Slice the impact matrix horizontally. |
|
For metrics that are useful in near real-time, and are worth pennies… Hand them over to machines. There is no reason any human needs to be involved in them. In fact, if a human gets involved, you are going to delay the impact because a human will only slow decision-making.
Then, as the size of impact increases, so does the title/responsibility of the individual who needs that data.Â
Reflect on the CMO and VP dashboards your analytics team/agency is publishing.
Do they contain the KPIs in the top most layer?
If they don’t, are you taking people being paid extraordinary compensation and using their time to make decisions worth pennies or a few dollars?
The fix is straightforward, swap metrics from Solid Foundation to Advanced.
Oh, and since the top layer data takes time to become useful… You will auto-kill the daily “CMO Dashboard” you are sending out. Save yourself, your organization, and your CMO so much grief!
Another pernicious problem analytics teams face is: Who should do all the analytics work that needs to get done?
Often you have core IT teams, an agency (or two), in larger companies smaller analytical teams in different divisions, and likely a centralized strategic measurement team to ensure your company stays on the edge of innovation.
To ensure all these resources are organized against the optimally responsibilities… Slice the impact matrix vertically. |
|
Again, anything that’s useful in real-time needs to be handed over to machines. If you hook this up with even a basic automation effort across platforms… You are going to die of delight. (And, all the humans you saved from brain-optional tasks will worship you.)
I’ve worked in every team you see above. These days, I lead a global core strategic analytics team. Almost the entirety of our time is spent on the hardest possible questions to answer - the Advanced cluster.
Please reflect on who does what in your company. Are the most expensive resources solving the hardest problems? Or, they spend their days reporting impressions and visits?
Now you have a roadmap to ensure your resources are aligned optimally.
One last bonus picture… The automation and machine learning zone… |
|
It is of value to reflect on your current deployment of machine learning and automation. Check if they cover the areas identified in gray above.
If yes, celebrate!
If no, make plans to get there.
Without escaping the zone in gray above, it will be impossible for you to leave behind low-value analytics requests and find time to focus on the analytics that matters. Living in the gray zone is also non-exciting for your humans - they are spending time working on things that are useful in real-time and not worth much to the business.
Algorithmic automation will help you buy your freedom, it will bring you joy.
[TMAI Premium members, see “TMAI #271:  The Future of Analytics.” for more on algorithmic intelligence and intelligent automation. If you can’t find it, ping me.] | Bottom line.
We have more data than perhaps God intended us to have.
We have more tools, often free, than should be available to any human.
Yet. Most companies are data rich, information poor.
The problem is not the ink. The problem is the think.
The impact matrix is a framework that help you apply super smart think, to make difficult choices, and chart an entirely new, super productive, future for yourself, your team, and your company.
Carpe diem.
Avinash. | Committed to investing in your professional growth? Upgrade to TMAI Premium here - it is published 50x / year. It'll transform your salary. :) | |
|
|
[ Subscribe] Â |Â Â [ Web Version]Â Â |Â [ Unsubscribe]
                            ©2022 ZQ Insights  |  PO Box 10193, San Jose, CA, 95157, United States of America
|
|
|
|