The note that read beautifully and meant nothing
Day four of the close. The board pack needs commentary on why services margin came in 380 basis points under plan, and the FP&A analyst pastes the actuals into an AI assistant. Thirty seconds later there's a clean paragraph: "Gross margin compression was driven by elevated subcontractor utilization and project overruns in the implementation practice." It's grammatical. It's confident. It sounds like every variance note you've ever read.
And it's useless, because nobody can tell you which projects, which subcontractors, or whether the model just pattern-matched a plausible story onto a number it never actually reconciled. The controller asks "show me the bridge" and the answer is silence. Now you're doing the analysis you thought you'd automated, except you're doing it at 9pm with a draft you have to disprove first.
This is the trap with variance commentary specifically: the output is prose, and prose hides its sourcing. A forecast model is wrong in ways you can see. A variance note is wrong in ways that read fine until someone in the audit committee asks a follow-up. The IBM Institute for Business Value AI capabilities research is blunt about where value actually comes from — data quality, ownership, adoption, measurement — and a variance note touches all four at once. It has to bind actuals to budget to forecast to the operating reason behind the swing, then survive a reviewer who is personally accountable for the number. A fluent paragraph clears none of those bars on its own.
Measure the two things that actually move: traceability and rewrite rate
Forget "hours saved" as your headline metric — it's the easiest number to fake and the easiest to argue about. For variance notes, two measures tell you whether the tool earned its place.
Traceability rate: what fraction of generated explanations cite a specific source the reviewer can open — the GL detail, the project ledger line, the headcount file — versus explanations that assert a driver with no link behind it. Say a 120-person tech-enabled services firm runs commentary on 15 variance lines a month. If 11 of them come back with a clickable trail to the source record and 4 are confident hand-waving, your real ROI lives in those 11, and the 4 are where someone gets burned. Track the ratio. Watch it move.
Controller rewrite rate: of the notes the assistant drafts, how many ship roughly as-is versus get rebuilt from scratch by a reviewer. If your CFO or controller is still rewriting the narrative wholesale, you haven't saved a step — you've added a draft-to-disprove. The honest answer might sting in month one. It should fall by month three or the pilot is telling you something.
Both metrics depend on governance, and this is where finance teams underinvest. The NIST AI Risk Management Framework gives you the vocabulary that matters here: define the use case narrowly, name the failure modes (fabricated drivers, stale numbers, mismatched periods), and assign accountability for review. A variance assistant should always surface three things — where the number came from, who owns the explanation, and what it's unsure about and is flagging for the controller. The third one is the feature that separates a tool from a liability.
Then there's the part nobody wants to discover in an audit: where the commentary was assembled from. Variance notes get stitched together from spreadsheets, BI exports, finance emails, and last quarter's board materials. Microsoft 365 Copilot's data protection architecture shows why permission boundaries and an audit trail belong in the ROI model from day one — not as a compliance afterthought but because a tool that can quietly pull a restricted comp file or a superseded forecast version into the close package isn't a productivity gain, it's an incident waiting for a date.
Pick one variance line, run it next close
Don't roll this across the whole P&L. Pick the single variance category that eats the most reviewer time — usually labor/utilization or a specific cost center that swings every month — and run AI commentary on only that line for one full close. Capture your traceability rate and controller rewrite rate against the manual baseline you remember from last month: the time spent chasing owners, the unsupported explanations, the board-pack rework, the close calendar slipping while people argue over a driver.
If both metrics improve and the controller starts trusting the draft, expand to a second line. If the controller is still rebuilding from scratch by close three, the lesson isn't "AI doesn't work for finance" — it's that your source data underneath that line isn't clean enough to explain, and you'd fix that before any tool helps. The McKinsey State of AI 2025 data lines up with what we see in the field: the gap between firms getting measurable value and firms running expensive demos is rarely the model. It's whether the workflow underneath was disciplined to begin with.
Put numbers to your own close before you commit budget. Run the AI ROI Calculator against your variance-note workload, take the AI Opportunity Score to see where finance commentary ranks against your other candidates, and if you want help designing the pilot so the metrics mean something, that's what Human Renaissance AI transformation services are built for.
