Deployment readiness for regulated AI agents
The agent is the easy part.
Every regulated deployment dies at the same three gates. What each gate asks depends on the supervisory regime, and the answers live in documents no institution publishes.
Select a regime: the gates stay the same, the questions do not
These are the questions, not the answers. An outside system can retrieve which regime binds an institution. It cannot retrieve how that institution will decide. That is a measured result, not an opinion: ten institutions, nine supervisory regimes, the same outcome every time. The asymmetry is the whole problem, and the reason the work has to happen inside the perimeter.
01 · The finding
Ten institutions, nine regimes, one boundary.
bfsi-deployment-qualifier takes the name of a bank or insurer and returns a brief on whether an agent deployment can realistically land there, written from the buyer's side of the table. Every regime it names was retrieved during that run with a source attached, not assigned by me.
Every citation is checked against a ledger of URLs actually returned by search during that run. A URL that was not retrieved fails, however plausible it looks. Grounding is a property of the run, not of the document.
Building it on GitAgent is what makes that inspectable. The agent is a repository, so its rules are a diff, its enforcement is a file you can open, and the run it rejected is a committed artifact rather than a screenshot. Auditability is not bolted on afterwards. It is what the format already was, which is the property a second-line reviewer is actually buying.
It caught a real fabrication on the NatWest brief. The agent listed a regulator's page as its source, and the link looked entirely legitimate: correct domain, plausible path, describing a document that genuinely exists. It had never opened that page. It had assembled the address from the shape of other URLs on the same site.
No reader would have caught that. The ledger caught it immediately, because it compares every citation against the pages actually retrieved during that run. The rejected brief is committed to the repository on purpose.
Asked about an institution that does not exist, it finds the real one with a near-identical name and reports them as separate entities rather than merging them.
The lesson generalises. Every constraint written as prose was bent at some point during the build: the output shape, the search budget, the citation. Every constraint compiled into a check held. A second-line reviewer cannot accept "we asked the model nicely", which is why enforcement lives in code here and not in the prompt.
The most useful thing it produced was a negative result. The qualification verdict returns the same confidence for every one of the ten institutions, from a $712m credit union to JPMorgan Chase. I looked for a prompt defect, found one, fixed it, and re-ran the two extremes. Nothing moved. That is not a tuning problem. What an institution is (licence, scale, supervisory perimeter) is published. Whether its model risk function treats a non-deterministic agent as a model, what its residency policy permits, which committee signs and above what value: none of that is public, anywhere, for anyone.
So the deliverable is not the verdict. It is the list of questions only the sponsor can answer, which is exactly what you walk into the room holding.
02 · The thesis
The framework layer commoditises. The deployment layer does not.
Agent orchestration is converging fast. The frontier labs are shipping their own runtimes with tool permissioning, evaluation and governance attached, and those will be free, well documented and default. A platform positioned on being the better framework is defending ground that is being given away.
What does not commoditise is the part that resisted automation.
IRDAI and RBI produce different blockers for the same use case in the same country. SS1/23 and SR 11-7 ask different questions of the same model. A Swiss private bank and an Australian mutual fail at different gates for reasons neither publishes.
That knowledge is per-regime and per-institution, and it does not generalise from a model's training data because it was never in that data. It is a deployment layer, not a framework feature.
The boundary measured above is what makes that position durable rather than merely correct. If the deciding facts cannot be retrieved from outside, they can only be obtained from inside: from a named sponsor, in a meeting, one institution at a time.
Whoever captures those answers as structured data, rather than in a solutions architect's memory, ends up holding a map of how these decisions actually get made, per regime and per institution type.
That map cannot be scraped, and it cannot ship inside someone else's runtime, because it was never public to begin with. It is the one asset in this stack that gets more valuable with each deployment instead of less.
It is also the only one a competitor cannot acquire by shipping a better abstraction.
Two constraints, stated before anyone else raises them. The raw answers are client-confidential, so what accumulates is anonymised pattern rather than record. And it compounds only if it is captured into the product. Knowledge that lives in people is a consultancy: it scales linearly, and it leaves when they do.
So the product question is not how to build better agents. It is which questions to bring to the first meeting, how to capture what comes back, and how to make the next engagement start further forward than the last. That work sits above the framework, it is where the difficulty actually lives, and it does not arrive free in someone else's runtime next quarter.
03 · Scope
What this is not.
- Not a productA working artifact built in a day to test an argument. It is not hardened, and I would not put it in front of a client as it stands.
- Not adviceThe gates above are the questions a second-line function asks. They are not legal or regulatory advice, and they are not a substitute for the institution's own assessment.
- Not a verdictThe qualifier cannot make the deployment call from public sources. It says so in its own output, and the field would be cut in a second version.
- Not automatedThe questions that decide the outcome are answered by a human sponsor inside the institution. The system's job is to know which questions to bring.