# Nobody Asked for a Chatbot

> Teams are building their own AI analysts, and the data modeling underneath is still the hard part. But in my experience, adoption depends on how findings reach people. Most users want answers waiting for them, power users want analyst-grade access, and almost nobody has asked me for a chat box.

By Cody Bakken · 2026-10-16 · https://codybakken.com/writing/nobody-asked-for-a-chatbot/

This week's dbt Roundup,
[Open Data Infrastructure in the age of AI agents](https://roundup.getdbt.com/p/open-data-infrastructure-in-the-age),
makes the case that companies should build their own AI analysts on open data
infrastructure instead of buying one off the shelf. Tristan Handy's version of
the argument is the one I'd underline: most of the work is already done if your
data is in one place, modeled well, and documented. The MCP servers and the
model choice in the middle are, in the newsletter's words, not that big.

I agree with that, and I'd go further: data modeling matters more with agents in the picture. Where I'd add
something is the last step, which the episode mostly skips: how the findings
actually reach the people who use them.

The default picture of an AI analyst is a chat box where anyone in the company
types a question and gets an answer. I've run analytics for a while now, and
I've logged several hundred requests. Not one of them asked for that.

## Three kinds of users, none of them asking for chat

When I sort the people who use our analytics by what they actually want, I get
three groups.

**Most users want the answer waiting for them.** They have the same questions
every week or every quarter. They don't want to phrase a question, judge whether
the answer is complete, and decide if they trust it. That's analyst work, and a
chat box hands it to them.

**Analysts want time to go deeper.** This is where agents have changed my team
the most, and I'll come back to it.

**Power users want access closer to what my team has.** Our CFO does their own
analysis, and I don't get in the way of it. What power users get is a
negotiation over permissions, data access, and tools. In my experience they
don't want a chatbot either. When they want an agent, they want their own.

That leaves chat squeezed in the middle. Most users don't want to ask
questions, and the users who do want something stronger. I think of chat as an
interface looking for a user: it's built for someone who wants to explore but
doesn't need real access, and I haven't met many of those people.

## Finding the repeatable questions

Every request my team gets goes into HubSpot ticketing. That started as a way
to manage a queue, but the log turned out to be the useful part. When you can
look across several hundred requests, the repeats are obvious.

The clearest one was some version of "how does product usage relate to
outcomes?" It came from different people with different wording, but it was
the same analysis each time. That's the signal I look for: a question that keeps
coming back is one that should already be answered before anyone asks. So we
built it once, properly, and put it where people could get to it without a
ticket.

The request log tells me what people know to ask for. Usage monitoring in our
analytics tells me what they actually use once it exists. I need both, because
those two lists are not the same.

## Agents let analysts go deeper

The article's best argument for agents is the long tail. Handy points out that
human analysts handle the expected questions well, and agents can take on the
open-ended exploration nobody has time for. He cites Ramp's research agent
growing to roughly 50 times the data team's question volume in six months.

Worth being straight about: Ramp's agent was a chat interface in Slack, open to
anyone. That's the strongest case against what I'm arguing here. My read is
that 50x the questions tells you the demand was real. It doesn't tell you how
many of those answers were right, or who checked them before someone acted. In
my setup, that checking is the analyst's job.

I agree, but I put the agent in a different seat. In my setup, the analyst is
the one asking the long-tail questions.

Our ICP work is a good example. Describing the ideal customer profile is easy.
Testing which traits actually line up with wins and losses is slow, because
every trait has to be checked against real deal outcomes. With agents doing the
legwork, my team could go much deeper on which traits held up against actual
win/loss data, instead of stopping after the first few cuts.

The difference from chat is who stands between the model and the decision. An
analyst knows when a number looks wrong, which join double-counts, and which
trait is really a proxy for deal size. The end user gets the finding after
someone with that judgment has checked it.

## Power users bring their own agents

Our marketing leader is a good picture of where power users are going. He has
his own agent, his own instructions, and his own flows, and he uses them to
drive campaigns. Today that runs on exports plus MCP connections to Google
Drive and HubSpot.

What he connects to is curated analysis my team has done, often analysis he
asked for. He never touches raw data. Our
[ESSA Level III study](https://www.derivita.com/derivita-essa-level-iii-study-lander)
is the clearest case.
My team did the research. He turned its findings into emails and web pages
through his own agent.

That changes how I read one of the article's warnings. The guests argue
against wiring agents straight to SaaS tools, because each system organizes
data around its own roadmap and MCP is just an access protocol. Our marketing
leader does wire his agent straight to Drive and HubSpot, and it works. It
works because what's sitting in Drive is analysis we already modeled, checked,
and wrote up. The context problem got solved upstream.

So I'd put the rule a little differently: don't point agents at data nobody has
curated. Curated views exist because analyzing data takes more than having
access to it, and a confident wrong number in a campaign costs more than
waiting a day for the right one.

## How I know it's working

The evidence is mostly usage. Our partner success managers tell me they're in
the internal analytics every day, and the thing they mention is the depth and
how fast they get answers. Customers say the same about our analytics platform:
the depth and how easy it is to use. Usage of the new analytics is up three to
four times over last year.

We got there without a chat interface. What drove it was answering the repeat
questions in advance, letting analysts go deeper, and giving power users access
they could build on.

## The part nobody asks for

There's a tension in everything above. I've said my users never asked for chat.
I'd also say the most valuable thing a data team does is find what nobody asked
for. The Ramp story is exactly that: demand nobody saw until the tool existed.

Here's how I resolve it. The request log and the usage data show you demand
people know about. Value nobody requested comes from someone with experience
looking at the data, keeping up with new techniques, and seeing what users
couldn't picture. That takes regular research into what's newly possible, and
it's a big part of my job.

Adding item response theory (IRT) analysis to our assessment reporting is my
best example. Nobody filed a ticket for it, and nobody would have, because you
have to know IRT exists before you can ask for it. It's a psychometric method
that estimates how difficult each question is and how well it separates
stronger students from weaker ones, so reporting can say something about the
questions themselves, beyond percent correct. That idea came from keeping up
with assessment research. It never showed up in the request queue.

A chat box hands that discovery to the user, who is
the person least equipped to do it. An agent in an analyst's hands multiplies
it.

## What's next

The direction I'm heading is the opposite of chat. Instead of waiting for
people to come ask, findings should go to them: push alerts, findings sent by
email, and findings that trigger a workflow directly. If a finding matters to
someone's job, they shouldn't have to come looking for it.

That's still the data team's job. The modeled data has to be right, and the
analysis has to be checked. The only thing that changes is the delivery.