A bank's 90-day deployment: moving AI customer service from demo to production

On day 90, the delivered numbers were: 71% adoption among customer service reps, average response time cut from 8.4 minutes to 3.1 minutes, and 63% direct adoption of AI-generated drafts. These weren't the demo numbers. Everything looked polished at the demo.

This regional bank's credit card support center had deployed a commercial LLM customer service tool. The proof of concept ran smoothly, and when management reviewed the presentation, everyone approved. Three months later, when the team arrived to assess, only about 20% of reps were using it daily. The system wasn't broken. It simply didn't fit how this bank worked.

Weeks 1 to 2: Listening to calls, not writing code

Observation came before building. Two engineers sat in the customer service center with headsets on, listening to calls and watching reps navigate five internal systems to resolve a disputed charge.

The actual problem wasn't the model. It was that 40% of inquiries involved a specific transaction, and that transaction data was locked in a 1990s mainframe the generic tool couldn't access. Reps didn't use the AI because it could explain general card policies, but couldn't answer 'Why was Mr. Wang's 3,200-yuan transaction blocked last Wednesday?' A tool that doesn't solve the actual question doesn't get used.

Learning this in two weeks was worth it.

Weeks 3 to 6: Routing RAG through the mainframe, not the FAQ

The knowledge retrieval layer was rebuilt. The 200-page FAQ previously loaded into the vector database became secondary reference material. Real-time transaction queries became the engine. When a rep opened a ticket, the system fetched the transaction status, dispute flags, and risk-control notes in deidentified form, added them to the context, and generated a response draft.

The first major obstacle appeared immediately. In version one, the AI automatically pulled the customer's full account data into the prompt. The compliance officer stopped the project cold at security review. Over-collection violated the principle of least privilege. Nine days were spent rebuilding the permissions layer so the AI could access only the specific transaction the rep was handling, with every read logged for audit. Slower, but necessary before launch.

The second obstacle was hallucinations of the dangerous kind. During testing, the model confidently produced a 30-day interest-free grace period that didn't exist. One false claim in financial services becomes a customer complaint or regulatory fine. The solution wasn't prompt tweaking. A rule was added: responses touching interest rates, fees, or compliance terms must quote directly from the knowledge base without rewording and must include the source reference. The AI saying I need to confirm is better than confident fabrication.

Weeks 7 to 12: Adoption by design, not by announcement

A good tool nobody uses provides nothing. Three factors pulled adoption from 20%.

First, the AI generates only drafts. Reps always send the message. This preserves human judgment and dignity, and prevents senior staff from seeing it as a replacement. It's a tool. Second, the interface was embedded into the ticket system they already used, not a separate tab. Every additional window reduces adoption. Third, each week the team pulled the top ten types of questions where reps edited AI drafts most and used that to refine retrieval and rules. The system visibly improved week to week. That's how trust develops.

Here's how the metrics changed over 90 days:

The deliberate tradeoffs

Some work wasn't done. The customer wanted fully automated responses during night shifts with no human review. The team recommended waiting. Automating disputed-charge and risk-control scenarios carries too much risk in the first 90 days. That conversation will reopen once there's more trust data. The 29% of reps not using the system daily mostly handle highly specific collections and legal work. Those shouldn't be forced into an AI system.

This project is detailed because too many deployments show impressive demos followed by quiet launches. The difference doesn't come from model power. It comes from whether someone actually sat in the field, plugged the AI into the legacy system, worked through compliance review, and stayed until reps trusted it. That's how field deployment works: engineers on site until adoption actually happens. A polished demo doesn't count. The test is whether people use it every day.