If buy-in and stakeholders are where you get stuck, ExpertEdge has 3,400+ Packt and Wiley books and courses in one place, sorted by role, leadership, stakeholder management and communication included. For this piece I’d start with Stakeholder Management and Influence & Persuasive Communication.
The yearly plan is $120, and with the code DATAGIBBERISH50 on expertedge.org it’s $60, so $5 a month. I get a small kickback if you sign up.
Your data team is overwhelmed with ad hoc requests, every number you send out comes back with a question about why it doesn’t match the last one, and somebody above you has decided the fix is to put Claude on the warehouse and be done with it.
Last week, in London, we released our AI-powered solution. I wrote a somewhat technical guide on how to build a bot like ours on my personal blog. But before going with that, I strongly encourage you to read this piece on AI for ad hoc reporting first.
I know it’s boring, but starting with AI is the wrong move. You need to start with a lot of hard work, and what you get for it is the thing the business wanted long before it wanted AI, stable and predictable reports.
The AI was the easiest part of our bot
Our bot is called Chandler. It runs on Omni and Snowflake, and it was an idea my coworker Prabhu and I came up with at the exact same time.
We solved analytics long ago, so slapping AI on top of our solid data platform was easy. What made it easy was having solid modeling and semantics in code.
It also helps that we have stable pipelines and dashboards that the whole business trusts, and a rota, so one analyst is always “on call” to answer the kind of question people now bring to Chandler anyway. Chandler is a thin layer on top of years of work nobody outside the data team ever saw.
Claude on a messy warehouse is one more person writing the query their own way
Putting Claude on a messy warehouse doesn’t clear the ad hoc queue. Without written KPI definitions, Claude reads your tables, guesses what an active user or revenue means, and writes the query its own way, which is what your teammates were doing already, so the numbers still don’t match.
I talk to data people who are trying to build an AI solution that would offload ad hoc work, and most are doing it wrong. They have no foundations. They pull some data, but don’t do any proper modeling.
Whenever a stakeholder comes, a random team member picks the request, writes a query, and sends a CSV. The person who picks the request is not necessarily an analyst or an expert, and has no guide on the KPIs, so everyone writes the query the way they think is correct.
Requests pile up and stakeholders are unhappy, so these teams decide to slap Claude on top of their warehouse, hoping it will figure things out. It doesn’t work.
Claude has no guide on the KPIs either, so it’s one more person on the queue, the one who never sleeps, never says “I’m not sure” and gives a different answer at 3am than it gave at 3pm.
If your team works like this, the bot makes your queue faster at producing numbers that disagree.
The usual fixes are an intake form, asking for the business reason and turning repeat requests into reports. They cut the volume, and I’d do all three, but the person who picks up the request still writes their own version of revenue.
We run a rota too, and it works for one reason, the on-call analyst and Chandler read the same definitions. Without the definitions, the rota only decides who guesses this week.
Different numbers cost you their trust
Stakeholders are unhappy because of the speed, and also because the numbers are always different. Without proper modeling and reports, you get different numbers every time. Eventually, stakeholders stop trusting your AI, and therefore you.
If you’re the strongest engineer on a small data team and you built the bot, it’s your name on the number, and Claude takes none of the blame. The head of finance who gets one revenue figure on Monday and another on Thursday stops asking you and starts a spreadsheet of their own, which gives the company a fourth version of revenue.
Rebuilding that trust takes a hell of a lot longer than building the bot did.
The foundations are phase one of your AI project
If you want the boring work approved, call it phase one of the AI project, because it is. I think leadership that wants AI this quarter will fund the first phase of an AI project long before it funds a modeling initiative with no AI in the name.
Nobody gets promoted for writing definitions, and they’re boring as shit to write, which is why most teams skip straight to the bot.
I’ve been on the wrong side of this too, because semantic layers used to feel like a gimmick to me, a nice idea that never justified the overhead, and I changed my mind. When every leader pulls revenue from a different place, a shared definition in code is the only thing both people and machines use.
One written definition per KPI
Nobody should be able to invent their own metrics. Make sure the entire business has a shared understanding of what a user is and how you measure revenue, and document your processes.
If different teams use separate words for the same thing, explain these. In my experience, data teams consider a ton of things to be “easy to understand” or “obvious”. This is never the case.
A great tip is to keep snippets of commonly used queries that answer questions. My team has a GitHub repository called data-ops-snippets.
A model that grows with the business
I often hear that in the age of AI, data modeling is dead. This is not true.
A good data model represents the business you have today and grows together with it, so the definitions you wrote down still mean something after the next reorg or the next product launch. It’s the most important component you need, and it’s the one Chandler reads every time somebody asks it a question.
If you want to go further on modeling, Joe Reis has a recently released book on it, Mixed Model Arts.
A handful of reports for the KPIs that matter
Once you have definitions of your objects, processes and KPIs, and once you put everything into modeling, you need to stop running queries manually and build some reusable reports.
Do not build everything you are being asked to.
Pick a handful of your most important KPIs, like bookings, number of users, some product usage, and whatever your company’s north star is.
Dashboards are great, but even some automated Google Sheets exports are fine for a starter. Make sure people have access to these reports any time they need them.
The bot is a sense check, and the dashboards stay
We built Chandler so our stakeholders could ask more wild questions they don’t really want to ask people. We wanted to give them a quick sense check of things they don’t want to waste a real person’s time on.
The bot never replaces a real analyst. It doesn’t give deep dives and still needs someone to check responses. We always disclose that numbers may be wrong, and if people need something spot-on, they contact us on the #analytics-questions-and-alerts channel.
No matter how strong your AI solution is, people need stable reports to cross-check the AI numbers.
So if somebody is pitching your bot to leadership as a way to need fewer analysts, they’re pitching bullshit, and you’re the one who’ll be asked to deliver it. Our rota still runs, and the dashboards are still where every number gets checked.
What got our CEO to yes was a guide on when to use what
We did have an approval fight, and it was with our CEO. We needed to make sure we had provided proper guidance on when to use what, and that the tool only returns governed numbers and definitions. It was all great once we shared the guides with him and promised to share them with the announcement for the tool.
What settled it was having the rules for when to use what written down before launch.
So write yours before your bot goes live, and make it say which question goes to the bot, the dashboard or the analyst, and what people should do when the answers disagree.
Final thoughts
I think a lot of the rush to put AI on reporting comes from fear. Every week another company announces its AI analyst, and it starts to feel like everybody has a bot except you, and your team is falling further behind every day you spend writing definitions nobody asked for.
If that’s what’s under all of this, most of what you read about AI in data is wrong, and so, no, you are not behind.
The part that decides whether a bot works at all is the boring part, and you can start it this week. Chandler was easy for us because that work was already done, and nobody outside the data team ever watched it happen.
Your stakeholders don’t really need AI. They need quick access to numbers they trust.
Speed matters, and I won’t pretend it doesn’t, but a fast answer that changes between Monday and Thursday costs you more than a slow one, because it spends the trust you need to get anything else approved.
So before anyone writes a single prompt, ask three people on your team to write down how they calculate revenue, without talking to each other first. One answer means you can start building. Three answers means you’ve just found your phase one, and you’ll have a damn good reason to get it funded.
—
Until next time,
Yordan
PS: If you want to be the person leadership trusts with the numbers, and the one they think of when the next title opens up, I write a deeper piece for paying readers every Wednesday, and they also get the recordings of my course Design Your Promotion.
Asking the bot the same revenue question five times and diffing the SQL is a cheap way to find those missing definitions. Wherever the queries disagree on filters or joins, that's the KPI nobody has written down yet.