The real cost is your three people who actually know
In a technology-enabled services firm, there are usually three to five people who carry the company in their heads. Which clients have the weird billing arrangement. What the implementation runbook actually says versus what the doc claims. Why you never deploy on a Friday for the financial-services accounts. Every time someone interrupts one of them with a question the company has already answered, you pay twice: once for the asker's stalled work, and once for the expert's broken concentration.
That is the line item an internal knowledge assistant is supposed to move. So measure that, not "average search time." Before you roll anything out, baseline the boring stuff for two or three weeks: how many times per week do people DM a senior engineer or the ops lead with a "quick question"? How often does a search end in a retry or a shrug and a Slack message instead? How many answers people act on turn out to be stale? Count avoidable escalations — the ones that bounced upward only because the answer was buried behind the wrong folder permissions or a doc nobody trusted.
The IBM Institute for Business Value research on AI capabilities is worth reading here because it makes a point most knowledge-search pitches skip: value tracks capability maturity, not library size. A 4,000-document corpus that people don't trust is worth less than a 200-document one they do. The asset isn't the volume of answers. It's whether someone bets their afternoon on what the assistant tells them.
Wire the permissions before you wire the dashboard
Knowledge search has a failure mode that a chatbot for marketing copy doesn't: it can confidently surface something the asker was never supposed to see. In a services business, that's a client's contract terms leaking to the wrong account team, or restructuring notes showing up in a search by someone three desks over. The Microsoft 365 Copilot data protection architecture exists precisely because identity, permissions, content boundaries, and audit logging decide what the assistant is even allowed to retrieve. If your file permissions are sloppy today, the assistant doesn't fix that — it indexes the sloppiness and serves it faster.
So your ROI model needs two columns most people forget. First: permission exceptions — cases where the assistant returned something it shouldn't have, or refused something it should have. Second: restricted-source misses — questions it couldn't answer because the source was correctly walled off, which is a feature, not a bug, and tells you where a human handoff is mandatory. A model that only counts faster sessions and ignores these will look great right up until the incident that ends the pilot.
The NIST AI Risk Management Framework gives you the scaffolding to do this without turning it into a six-month committee exercise: define the use case narrowly, map the failure modes, set controls, and name an owner for answer quality. Concretely, that means three rules before launch — every answer cites its source, low-confidence answers say so instead of guessing, and anything touching live client context, current policy, or restricted material routes to a person. Decide those rules first. They're cheaper to install than to retrofit after someone trusts a wrong answer.
The signal that tells you to expand — or stop
Here's the read after 90 days. Pull your baseline numbers again. If senior-expert interruptions are down, time-to-an-answer-people-act-on is shorter, and stale-answer corrections are dropping, you have something real — expand it to the next domain. But watch for the tell that matters most: are people still bypassing the assistant for the same handful of topics? If the deployment team keeps DMing the lead architect about the migration runbook no matter what the assistant says, the interface isn't the problem. The source is wrong, out of date, or owned by nobody — and no amount of better search fixes an answer the organization doesn't actually trust.
When that happens, the fix is unglamorous and human: assign an owner to that document, repair it, and re-measure. That's the difference between a knowledge assistant that compounds and one that quietly becomes the tool everyone learned to ignore by month four.
If you want to put numbers to your own baseline before you commit budget, run the AI ROI Calculator and the AI Opportunity Score, or see how Human Renaissance AI transformation services scope a knowledge-search pilot down to one governed library worth defending.
