Instrumenting an internal AI assistant in production: what to log on every run, how to detect silent failures and quality drift, and how to structure a rollback when a prompt change goes wrong

This article is about instrumentation: what to log every time your internal AI assistant runs, how to catch it failing silently, and how to roll back a bad prompt change before it quietly wrecks a week of work. Most teams ship the assistant, celebrate the demo, and only build logging after something has already gone wrong. That's backwards, and it's the single most avoidable mistake in this whole category.

Here's the thing about internal AI tools that nobody puts on the launch slide: they don't fail like normal software. A broken API throws an error. A broken prompt just... answers confidently wrong. No stack trace, no red banner, just a slightly worse answer than yesterday, delivered with the same chipper tone as always. Your assistant gets dumber, one prompt tweak at a time — like a photocopier of a photocopier, each version a little blurrier than the last — and nobody notices until someone in finance asks why the expense-report bot approved a $4,000 "team building" charge for what turned out to be a jet ski rental.

I once worked with a team that discovered this the hard way. Their assistant had been quietly truncating answers for eleven days before anyone noticed, because the truncated answers were still grammatically complete sentences — just missing the one caveat that mattered. Eleven days. Nobody looked because nobody was logging, and nobody was logging because the demo had gone great and everyone assumed great demos age like wine instead of milk.

Log Every Run Like You'll Need It in a Deposition

You need a record of what happened, every single time, for every single call.

At minimum, capture these on every run:

• The full prompt sent to the model, including the system prompt, not just the user's message. If you only log the user's question, you'll have zero idea whether the system prompt someone edited on a Friday afternoon is what broke everything on Monday.
• The model's raw response, before any post-processing, truncation, or formatting cleanup touches it.
• Model name and version string. Providers update models under the same name more often than people change their car's oil. If you don't pin and log the exact version, you can't tell if a quality dip is your prompt or their update.
• Token counts, latency, and cost per call. These are your canary in the coal mine. A sudden jump in output tokens often means the model started rambling or looping, and you'll see it in cost before you see it in complaints.
• A trace ID that ties the run to a user, a session, and a timestamp. When someone says "the bot gave me a weird answer yesterday around lunch," you need to find that exact call in under two minutes, not spend an afternoon grepping logs like it's 2009.

Store this somewhere queryable, one that won't grow forever and become unmanageable. A basic table in Postgres or a logging platform like Datadog or Honeycomb works fine. The tool matters less than the discipline of actually writing to it every time.

Silent Failures Are the Real Enemy

A loud failure is a gift. The assistant times out, the API returns a 500, someone gets paged, it gets fixed. Fine, annoying, but fine.

The dangerous failures are quiet. The model still returns 200 OK, and it still writes a full paragraph. It just happens to be wrong, or vague, or subtly off in a way that reads fine at a glance and only falls apart when someone actually acts on it.

Some patterns worth watching for, based on how these tools tend to drift:

• Length collapse. The assistant starts giving noticeably shorter answers to the same class of questions it used to answer thoroughly. This often means it's hitting a token limit it didn't used to hit, or a system prompt change accidentally told it to be terse.
• Repetition and hedging creep. Watch for phrases like "I'm not entirely sure, but" or the same disclaimer sentence showing up across unrelated answers. That's usually a sign the underlying model swapped versions or the temperature setting got touched.
• Confidence without content. This is the scary one. The tone stays helpful and assured, but the actual information thins out. A model that says "Great question! Here's what you need to know:" followed by nothing useful is worse than one that admits uncertainty, because a confidently empty answer is like a vending machine that lights up, hums, and takes your dollar, and then gives you nothing but the sound of a spiral falling and not catching.

The fix is a habit. Sample a percentage of real production runs weekly and have an actual human read them closely. Automated evals help you scale this later, but they don't replace a person noticing that the assistant has started answering every question about vacation policy with a slightly condescending tone since last Tuesday.

Building an Actual Quality Baseline

You can't detect drift if you never measured the starting point. That sounds obvious, and it is; most teams skip it anyway, because measuring quality feels like homework compared to shipping features.

Build a small, fixed set of test prompts, maybe 30 to 50, that represent the real questions your users ask. Run them against the assistant before every prompt or model change, and save the outputs. This is your regression suite: instead of checking that a button still clicks, you're checking that the answer to "how do I submit a PTO request" still resembles a correct answer to "how do I submit a PTO request."

Score each output on a few simple axes: correct, incomplete, wrong, or off-topic. A spreadsheet with four columns beats an elaborate scoring framework that nobody maintains past week three.

Compare the new batch against the old batch every time you change anything. If your "correct" rate drops from 90% down into the 70s after a prompt edit, you've caught the problem before your users did. That's the entire point.

Rolling Back When a Prompt Change Goes Wrong

Prompts are code, so treat them like it, or pay for it later.

That means version control. Every system prompt lives in a git repo, or at minimum a change log with timestamps and author names, so you can always answer "what did the prompt say last Tuesday at 3pm" without relying on someone's memory or a Slack thread that got archived.

When a rollback becomes necessary, and it will, structure it like this:

1. Freeze the current version immediately. Don't try to "quick fix" a bad prompt live in production while users are still hitting it. That's like performing surgery on a moving treadmill; someone's getting hurt and it's probably going to be you.
2. Revert to the last known-good prompt, the one that passed your baseline test suite. Restore the version you already trust rather than guessing at a fix.
3. Re-run the baseline suite against the reverted version to confirm you're actually back to good.
4. Document what broke and why, in plain language, in the same place as your version history. Future you, six months from now, will not remember that adding one clause about "being more concise" caused the assistant to start dropping entire steps out of instructions.

Root-causing while your assistant is actively giving employees wrong answers is a luxury you don't have. Roll back first, investigate second. This is triage, and nobody asks a bleeding patient for their full medical history before applying pressure to the wound.

The Boring Truth

None of this is exciting, and there's no clever trick here, no secret prompt engineering technique that makes the problem disappear. It's logging, baselines, and version control, the same unglamorous stuff that's kept regular software running for decades, just pointed at a new kind of unpredictable system.

Why do old-school ops engineers make such good AI babysitters? Because they've spent twenty years assuming the system is lying to them, and it turns out that's exactly the right instinct here too.

The teams that get burned are the ones that treat the AI assistant as fundamentally different from every other piece of production software they've ever shipped. It fails more quietly, and it never once tells you it's sorry.