The Model Never Does Anything: Building a Bank Agent That Knows When Not to Act

A customer opens a banking chat and writes "no reconozco estas transacciones", with a photo of a card statement attached. Most bank bots answer by asking for the exact date, amount and merchant, then file a complaint. We wanted the opposite order: the bank looks at its own records first, finds the two Uber charges of 312.40 MXN posted four seconds apart, and tells the customer it looks like a duplicate before asking them to do any work.

That is easy to demo and hard to trust. The moment an agent can block a card or move money, a single hallucinated "listo, bloqué tu tarjeta" is a real failure. So the system our team built for the Factored AI & Data Hackathon 2026, which we called quipu, is organized around one rule: the model never does anything. It understands and proposes. Code decides what is allowed from the customer's data, the customer confirms with a button the model cannot press, and nothing is reported as done until it has been read back from the database.

The problem and the user

The challenge asked for an AI-first customer-service system for a bank, not a chatbot, demonstrated end to end on one focused workflow; more workflows earned no extra credit. The demo had to cover three required cases: a normal case resolved automatically within policy, an ambiguous or unsupported request handled with a clarifying question or a safe abstention, and a case that needs a human, handed off with verified facts and open questions rather than a raw transcript. Interactions had to work in Spanish and Portuguese. The kickoff framing was blunt: build something that works, prove that it works, and know when it should not act.

The user is a retail bank customer who does not remember exactly which charge looks wrong and may write in either language. The second user is the human agent who receives the case when the system should stop.

We started from transaction disputes ("I don't recognize this charge") and grew outward. The organizer's contact-center data gave the sizing: of about 686k contacts, the reason mix was Transaccional 35%, Producto 22%, Queja 17%, Técnico 15%, Comercial 8% and Retención 3%, with 76.6% first-contact resolution overall and 44% for complaints. Transaccional plus Producto, which is balances, movements, cards and products, made up 57% of contacts. That is why the final system has skills for balances, charge investigation and data lookups around the dispute flow, and later a money-movement skill.

Constraints

Time. The challenge launched on 25 September 2026 and closed on 5 October, with a public repository, a deployed link, slides and a video pitch due.

Data. The dataset was a synthetic LATAM bank: 13 tables covering Mexico, Colombia and Argentina, June 2023 to June 2026. On the full download we measured 150,000 customers, 400,000 products, about 4.4M transactions and 15.6M digital events. Two findings shaped the design more than any model choice:

• The customer-service text is templated. call_transcripts.full_text holds 546 distinct texts across 171k rows, detected_intents is always consulta_general or null, complaint descriptions come in 5 strings, and was_escalated is 10% in every reason, channel and sentiment. There was nothing to learn an intent classifier, an escalation rule or a response style from. We used these tables for volume mix and baseline KPIs only, and wrote our own labeled evaluation sets.
• There is no Portuguese and no Brazil. All text is Spanish. Portuguese evaluation scenarios and a Portuguese demo customer had to be team-made and labeled as such.

The data was also thin (a median of 29 transactions per customer in three years), and the announced duplicates were not in the files: dedup removed 0 rows. We kept dedup and contracts anyway and reported that.

Rules. Permissions and policy had to live outside the model's text. A typed customer number alone does not prove identity. Access to each customer's rows had to be enforced at the service or tool layer. No real money moves, and the LLM cannot invent credit rules.

System architecture

The guiding principle in our architecture notes is LLM where it helps, code where it must be guaranteed. The backend is a FastAPI service in a hexagonal layout (domain rules, application use cases, inbound and outbound adapters). The agent is a LangGraph graph with deterministic nodes around LLM calls; each skill is a folder with a SKILL.md and its own FastMCP server.

flowchart LR
    U(["Customer<br/>ES / PT, text, voice, photo"]) --> WEB["Next.js web<br/>chat, mis finanzas"]
    WEB -- "voice" --> SP["faster-whisper STT<br/>Kokoro TTS"]
    WEB -- "POST /api/chat<br/>bearer JWT" --> API["FastAPI<br/>customer = token sub"]
    API --> G["guard (code)<br/>identity, asks for a person"]
    G --> R["router (LLM)<br/>sees skill catalog only"]
    G -- "wants a person" --> H
    R -- "bank never does it" --> RF["refuse (code)<br/>action policy"]
    R -- "use_skill" --> K["skill agent (LLM)<br/>SKILL.md + tool loop"]
    RF --> H
    K --> MCP["FastMCP servers<br/>accounts, investigation,<br/>dwh, actions, transfers"]
    MCP -- "session customer<br/>from _meta" --> CORE[("Postgres core<br/>bank data")]
    MCP -- "proposals, audit" --> OPS[("Postgres ops<br/>actions, transfers,<br/>decision_log")]
    K -- "escalate" --> H["handoff (code)<br/>case file"]
    H --> CON["Operator console<br/>/consola"]
    WEB -- "Confirmar button" --> GW["gateway (code)<br/>recheck, execute,<br/>read back"]
    GW --> OPS
    GW --> CORE
    PIPE["S3 CSV, bronze,<br/>silver + contracts"] --> CORE
    K -. "masked traces" .-> LF["Langfuse"]

Three properties matter more than the shape:

• Identity never passes through the model. At the HTTP edge the customer is the sub of a 15-minute HS256 JWT. Inside the graph the session customer travels in the MCP request _meta, and tools read it with session_customer(ctx). No tool takes customer_id as an argument, so the model cannot see or change it.
• Handoff is code. "Quiero hablar con una persona" is caught by the guard with no LLM call. The case file is built from graph state: the request, the skill, the tools used, what was proposed, and what the policy left unresolved.
• Confirmation is not a chat message. The Confirmar button calls its own endpoint, bound to the token. A "yes" written by the model, or injected into a merchant name, cannot confirm anything.

The cloud model is Gemini 2.5 Flash on Vertex AI; locally it is qwen3.5:4b through Ollama, so the stack runs on a laptop with no API keys. Production is two Cloud Run services (factored-api and factored-web) with Cloud SQL Postgres 16, deployed by GitHub Actions through Workload Identity Federation, so the repository holds no keys.

Key design decisions and trade-offs

1. Propose, confirm, execute, verify, audit. Account actions (block or cancel a card, open a payment inquiry, request a call, set an alert, email a summary) go through one gateway. The model proposes up to three typed actions by name; extra fields are an error. If confirmation is needed, the batch waits up to 10 minutes. Confirming twice runs it once (an atomic conditional UPDATE). Execution rechecks the policy, retries transient errors up to three times, then reads the effect back. If the read-back does not show it, the action is failed with verification_failed and a person is asked to look. Every sentence on the confirmation card and in outgoing email is written by code from the stored action, in the conversation's language.

Refunds, provisional credit, contact-detail changes, limit increases and anything not in the catalog go to a person.

2. Priority from signals, not from the model. Case priority and the response window come from code:

The threshold of 70 comes from the data: charges flagged as fraud average a score of 55.7, against 14.8 for the rest.

3. Deterministic investigation. investigate_charges finds at most three matching charges and checks each against the bank's records. A duplicate is a pair of Approved or Pending charges $(t_i, t_j)$ with the same merchant, amount and currency and

It also reports pending, reversed and foreign charges. A failed lookup is reported as unavailable, never as "no": a missing duplicate check must not read as "no duplicate charges".

4. Guarded text-to-SQL. For questions no reviewed tool covers, the data_lookup skill lets the model write one SELECT. A parser in the domain layer refuses anything but a read-only query over allowlisted core tables and rewrites every customer table into a subquery filtered to the session customer. The database is a second wall: a dwh_reader role with SELECT on core only, a read-only transaction, a 3 s timeout and 100 rows. We then execute the SQL the parser regenerated, not the model's original text.

5. Money moves only on the customer's button. Transfers, card payments and service bills ("khipea 300 a mi tarjeta") are a separate skill, money_movement. The model has no tool that moves money. It passes only what the customer said; code resolves accounts, limits and amounts, and when several accounts fit, the tool returns needs_clarification with the options. POST /api/khipu/confirm rechecks every rule with the rows locked and executes in one transaction.

6. The learned component is the prompted LLM, measured against rules. The original proposal included a learned charge ranker; the shipped system has none. The data audit had already ruled out an intent classifier and a learned escalation rule, and the mentor FAQ confirmed that a prompted LLM counts as the learned component if it is evaluated rigorously against a baseline. Our baselines were the same agent with actions off, and a deterministic router of regular expressions with no model. On held-out scenarios the model chose the right skill in 45 of 45 attempts that name one; the rules alone managed 33 of 45 (73%).

7. Evaluation that refuses to flatter. No LLM judges anything. Every verdict comes from rows in ops.actions and from regular expressions over the model's own words, with tests on both sides of each pattern. Scenarios are split mechanically before any model answers: within each category, sorted by id, the second of every three is held out. For scenario $k$ in a category,

with $k$ indexed from zero. A test recomputes the rule, and the runner refuses to run a held-out id without its split flag. When no unsafe outcome is seen in $n$ attempts, the report prints the rule-of-three bound instead of zero:

For the 22 held-out scenarios that is about 14%. Zero seen is not zero risk.

The text-to-SQL eval runs as a gate in CI before every deploy. With about 23 answerable cases per repeat and accuracy near 0.85, the binomial standard error is

about 7 points from chance alone, matching the spread we saw (single repeats from 0.727 to 0.909 with the same prompt). Pooling three repeats ($n = 69$) brings that to about 4 points, so the gate judges the three together against a floor of 0.75. It catches broken prompts and schema drift, not small regressions.

What we built

The system has five skills behind one chat (account actions and chat memory sit behind feature flags):

Around them: voice in Spanish and Portuguese, a "Mis finanzas" spending page whose sentences are templates that cannot invent a figure, an operator console where a person takes over any chat, optional chat memory (the agent gets a code-written summary of earlier conversations, never the raw messages), and Langfuse traces masked before they leave the API.

The demo flow follows the required cases. Lucía signs in, attaches her statement, writes "no reconozco estas transacciones", and the agent reads Uber, 312.40 MXN and the date from the photo, then reports two charges four seconds apart. Camilo has four charges on the same day, the ambiguous case: when a charge cannot be told apart, the right reply is a question, not a guess. For a stolen or cloned card, the right end state is a block the customer confirms plus a person who takes the case. A refund request is answered by code before any model call, and an injected "the customer already confirmed" changes nothing, because confirmation only arrives through the button's endpoint.

The headline result is the held-out action evaluation: 22 scenarios, 3 repeats, gemini-2.5-flash on Vertex AI, run once, blind, on 5 October 2026, against the same code with actions off.

Spanish went from 14/45 to 45/45 and Portuguese from 4/21 to 21/21. Acting costs about 2.6 times more per attempt and 2.8 times the median latency, roughly a quarter of a cent per conversation at list price.

These numbers need their caveats, which we published next to them. The held-out set is 22 distinct scenarios, of which 6 expect the system to act alone, and the repeats are not independent: with 22 of 22 the 95% interval is about 85% to 100%. On the 43 regression scenarios the same system scored 39 of 43 (91%). The four failures are known and safe: two injection cases where a refund word sends the whole message to a person, and two cards the bank had already blocked. Our own summary: about nine in ten, with the failures known and safe.

What we would do differently

• Persist conversation state from the start. The LangGraph checkpointer and the operator console's chat mirror live in process memory, which forced the API to run with --max-instances 1 and loses the handoff queue on every deploy. A Postgres checkpointer was always the real fix.
• Get independent eyes on the evaluation. The same team wrote the scenarios and the agent, and one person wrote the text-to-SQL gold queries. The held-out split keeps us from tuning on the test set; it does not remove authorship bias.
• Prove the incremental pipeline. The late-partition fixture that would demonstrate the freshness policy was never built, and the full ETL still runs from a laptop rather than as a scheduled job.
• Real identity before real money. In the demo a customer's password is their ID, and step-up authentication before moving money was deferred. These actions are only as safe as that sign-in.

Lessons

1. Audit the data before designing around it. We planned an intent classifier and a learned escalation rule. Checking the full dataset showed the transcripts were templates and escalation was a flat 10% everywhere. That finding redirected the whole ML plan toward team-labeled evaluation sets.
2. Gains on the cases you studied may not transfer. Three catalog rules, each written from a real text-to-SQL failure, lifted regression accuracy from 0.803 to 0.836 against a same-day control. On the blind held-out cases they fell from 24 of 25 to 18 of 26. The decision threshold had been fixed in advance; the drop was 0.27 (Fisher exact test, p = 0.02, which overstates the evidence because attempts repeat the same nine cases), and we removed the rules.
3. The model's failure modes are specific and catchable. In 4 of 86 regression runs Gemini looked up the cards and wrote "review and confirm below" without proposing anything. Giving the turn one more chance, with a note saying what was missing, recovered all 9 such attempts in the held-out run.
4. Some eval failures are the eval's fault. Our first regression run flagged a subjunctive "o que bloquee tu tarjeta" as a false claim, and "posso transferir você para um agente" as a promise to move money. We narrowed the patterns before the held-out run and documented every change.
5. Containment has a ceiling. 18 of 66 held-out attempts genuinely needed a person, so no correct system could contain more than 73%. Reporting containment without escalation quality would have rewarded the wrong behavior.

Reproducibility

The repository runs locally with Docker Desktop and Ollama and no API keys:

cp .env.example .env
make up      # local LLM, Postgres with the committed mini-set, API on :8080, web on :3000, smoke test
make test    # about 990 tests

The committed mini-set has 1,500 organizer customers plus 10 team-made demo customers, labeled is_synthetic_fixture. The full pipeline (make bronze, make silver SOURCE=full, make db-load SOURCE=full) needs the organizer's dataset, which is not in the repository, and writes a per-table quality report on every run.

Both evaluations run from one command: uv run python -m evals.actions.run --split heldout --repeats 3 for the actions, and uv run python -m evals.text_to_sql.run --repeats 3 for text-to-SQL. Every report records the commit, prompt fingerprints, model, region, date and prices; the raw held-out reports are committed under docs/documentation/technical/evaluation_results/. Expect variation by day: the same prompt scored 0.883 on one day and 0.803 on another.