Edge Case Shipping Guide: Exception Handling That's Easiest to Miss Before Enterprise AI Goes Live

Last month we took over a contract review Copilot that's been live for four months. The feature list looks solid: clause extraction, risk flagging, line-by-line summaries. The general counsel applauded during the demo. I pulled the usage logs. Monthly active users: 3 out of 11. The problem isn't the model. It's that those 3 users only upload standard procurement contracts. When they try a bilingual M&A agreement, a scanned document, or an old contract with handwritten notes, the system goes silent, generates nonsense, or treats attachments as the document itself. After two bad experiences, users went back to Word.

Here's the core insight: most of the work between demo and production lives in edge cases. A prototype that runs the happy path completes maybe 20% of the total launch work. The remaining 80% goes almost entirely to exceptions you deliberately avoided during the demo.

Why edge cases determine adoption, not accuracy

Demo logic asks: what can this do? Production logic asks: what will it do when it fails? The sample distributions are completely different.

A demo runs on curated cases (clean format, clear semantics, hand-selected examples). The real pipeline contains PDFs that are half tables. Numbers mangled by OCR. Three different ways of writing the same field. Users typing questions as fragments with no subject. Inputs where the question itself is ambiguous: does this contract even apply?

Trust works asymmetrically. One bad answer loses a user for ten correct ones. Adoption depends on what happens with the worst 5% of inputs, not on average accuracy. A 95% demo accuracy means nothing if that 5% fails silently with a confident wrong answer.

Step 1: Audit edge cases upfront: don't wait for them to blow up

Most teams wait for user reports to find exceptions. This costs the most because every report damages trust.

During the first week on site, we stress test. Pull 200 to 300 random samples from the customer's actual pipeline over the past six months (not the clean ones, everything) and run each through the system. Log every failure: wrong output, incomplete output, or crashes. This usually surfaces 40 to 60 distinct failure patterns in two or three days.

Auditing requires three answers: What does this input look like? What did the system output? What should the correct behavior be? Teams skip the third one most often, but it's crucial. For many edge cases, correct behavior isn't answering correctly. It's gracefully admitting uncertainty. A system that says I can't read this document completely, I recommend manual review of page 3 is more trustworthy than one that answers anyway.

Step 2: Prioritize: decide what to fix now versus what to block

After auditing you'll have a long, frightening list. The worst mistake is thinking fix everything before launch: you'll never launch. Prioritize on two dimensions: frequency and consequence.

The table's value isn't the categories themselves. It lets you and your customer explicitly decide what not to fix. The P3 row matters most: saying we're not fixing this, and we know we're not fixing this is more honest than pretending omniscience, and it makes scope deliverable.

The guardrails tier is underrated. You don't need the system to answer every difficult question. You need it to recognize which questions it shouldn't answer and defer cleanly to a human. This tier usually has the best return: a good decline-and-escalate mechanism often improves adoption better than three more months of training.

Step 3: Ship it weekly: make exception handling a rhythm

After prioritization, enforce discipline. Break P1 and P2 into a visible backlog where every item is a testable assertion: when input is X, system should do Y. Ship a batch every week. Fix one item, pick the corresponding test sample, verify you didn't break something else.

Don't say fixed and move on. Every edge case needs a test sample in the regression suite, permanently. AI systems update prompts or swap models, and old exceptions come back to life. Without regression tests, you're falling into the same hole repeatedly.

That contract Copilot with 3 active users out of 11 didn't need the model rebuilt. We spent three weeks adding detection for scanned PDFs, bilingual parsing, and applicability declination (three P0/P1 issues), then queued a dozen P2s for weekly iteration. By week six, monthly active users reached 9. What changed wasn't accuracy. Users now knew when the system would raise its hand and say I'm not sure about this one.

,

At Tenten, we judge frontline deployment on simple criteria: a polished demo doesn't count. It counts when it handles the dirtiest 5% of your real inputs and someone uses it every day. Our first move on site usually isn't to add features. It's to audit those edge cases nobody wants to touch one by one (the ones that are 80% of the work), prioritize them, and get them into the ship rhythm.