This note is background; current work is Web & eCommerce packages and US → EMEA market entry.
Most agent programmes do not die in the lab. They die after launch.
Sinch’s AI Production Paradox research (May 2026, n=2,527 senior decision-makers) found that 74% of enterprises rolled back or shut down a deployed AI customer-communications agent after a governance failure — not a stalled pilot, a live system pulled back. Among organisations that described their guardrails as fully mature, the rollback rate was higher: 81%.
It is vendor-commissioned research, so treat the absolute figures as directional. The direction is what matters, and the second number is the interesting one. The most plausible read is not that mature teams are worse — it is that they see more: they detect the incidents their peers never notice, and then discover that the controls they bought score prompts rather than filter rows.
The leading causes bear that out: PII or customer data exposure (31%), hallucination or brand risk (22%), and lack of auditability — unable to diagnose what went wrong (16%) (Sinch chapter). None of the three is fixed by a smarter base model. All three are decided at the boundary where an agent’s tools touch your data.
We call that boundary the Production Perimeter.
The second idea in this post is operational: time to production is time to the right data. Shipping an agent that can see everything is not faster than shipping one that can see the right rows. It only feels faster until the rollback.
Production is a governance gate, not a capability gate
The failure discourse still treats agents as capability purchases: pick a frontier model, wire some tools, ship. Our agent failure autopsy names the real pattern the Scoping Crisis — management and scope fail before the model does. This post is the complement: what you must enforce at the data seam once scope is narrow enough to ship.
Three truths operators learn in rollbacks:
- Agents inherit access paths. A service account with warehouse admin rights does not become safer because a language model sits on top. It becomes faster at reading — and leaking — what the account could already see.
- “What did it see?” is a data question. Regulators, security teams, and your own incident review ask for lineage and audit — which tables, which rows, which document chunks — not only the final natural-language answer.
- Observability without enforcement is a post-mortem tool. Traces that record a leak after 10,000 rows returned do not constitute governance. Gartner’s inaugural AI Governance Platforms Magic Quadrant (June 2026) weighted runtime intervention — can you stop a violation in path? — over static policy binders. That bar applies to agents querying Snowflake, not only to customer chatbots.
Models are good enough for narrow workflows. Access, audit, and drift controls are not.
A worked failure: the agent that saw the wrong tenant
Imagine a support agent with a SQL tool over tickets. The demo used a filtered view. Production pointed the same tool at the raw table with a shared service account. The model did what models do: it answered the customer’s question using the richest context it could retrieve — including tickets from another tenant, another country, another legal entity.
Nothing “hallucinated.” The model was accurate about data it should never have seen. Prompt injection was not required. A prompt firewall might have blocked a jailbreak; it would not have applied the row filter the warehouse already knew how to enforce for human analysts.
That is the Production Perimeter in one incident: the agent is a new principal on old data paths. If those paths were never fine-grained, the agent inherits the hole at machine speed.
Drawn, the perimeter is three gates in the request path — and one very common shortcut around them.
Scroll sideways on a narrow screen to see the full perimeter.
Three questions test any stack against that picture:
- Who is asking? Not “which app” — which named principal, for which declared purpose, expiring when.
- What may the tool do? Which fields, which operations, which rate limit — written down, not implied by a connection string.
- Which rows come back? If your agent’s service account can return a row that a human doing the same job could not see, you do not have a perimeter. You have a log.
Two vendor waves — and why they are not interchangeable
The market has sorted into two waves. That is useful, not tribal. Buy the wrong wave for your failure mode and you will still roll back.
Wave A — Agent and LLM security (new, prompt-shaped)
These companies are largely new. They grew with generative AI’s first wave: prompt injection, jailbreaks, toxic output, leakage in prompts, red-teaming before launch. They are necessary. They are not sufficient for production agents with tools.
| Vendor | Primary wedge | What it is strong at |
|---|---|---|
| Lakera | AI-native security platform | Runtime prompt-attack prevention, workforce AI discovery, low-latency guardrails |
| Preamble | Independent AI security assessments | Agent red teaming, tool-abuse scenarios, fractional security leadership for regulated deployers |
| Guardrails AI | Open guardrails library | Output validation, policy hooks developers embed in application code |
| NVIDIA NeMo Guardrails | Programmable rails | Dialog and tool-call constraints in NeMo-based stacks |
Wave A answers: “Was this prompt or response safe?” It does not automatically answer: “Should this agent identity run SELECT * on the customers table for this declared purpose?”
Wave B — Data access governance (old problem, new principal)
These vendors grew from warehouse and lakehouse FGAC, catalog, masking, and cross-engine policy. Agents did not create that problem; they added a non-human identity class that hits the same engines, faster, with worse audit stories when something goes wrong.
| Vendor / pattern | Primary wedge | What it is strong at |
|---|---|---|
| Native engine controls | Built into platforms you already run | Databricks Unity Catalog row/column policies, lineage, AI governance; Snowflake column security; AWS Lake Formation data filters |
| Apache Ranger | Open-source policy plugins | Engine-level enforcement hooks in Hive, Trino, and related stacks |
| Cross-engine access platforms (e.g. Trust3 AI) | Unified policy + agent control plane | Same logical FGAC across engines; purpose-based access; discovery/observe/secure between agents and data |
| Catalog / entitlement suites (Immuta, Collibra, and peers) | Policy and catalog at scale | Attribute-based access, classification, entitlement workflows |
Wave B answers: “Given this agent identity and purpose, what data may this tool touch — and can we prove it?”
Why Wave B has a structural advantage
Not because Wave A is useless — production stacks often need both. Because agents operationalise access:
| Hard production question | Wave A (prompt/runtime input) | Wave B (data access DNA) |
|---|---|---|
| Which rows may this agent see? | Not its native unit of work | FGAC: row filters at query time |
| Which columns may enter the context window? | Can redact PII in text after retrieval | Masking/encryption before retrieval |
| What grounded this answer? | Prompt/trace of the reply | Lineage from source → table/chunk → tool result |
| Same rule on Snowflake and Databricks and a lake? | Rarely | Policy sync / Ranger plugin / native constructs |
| Can we stop the query, not just log it? | Sometimes, at the LLM gateway | Runtime enforcement in the engine or trust layer |
- Lineage: Without catalog and lineage, you cannot reconstruct which retrieval corpus or SQL result grounded an answer. Unity Catalog tracks flow from source tables through models and services; that is the evidence chain auditors ask for when an agent misquotes a customer balance.
- FGAC at query time: Prompt filters do not row-filter. Fine-grained access — row filters, column masks, purpose-bound grants — must execute where the query runs: Ranger plugin, Unity Catalog policy, Snowflake masking, Lake Formation filters, or a synchronised policy layer. The pattern is documented across engines; Privacera’s FGAC overview is a clear secondary description of plugin-versus-native enforcement.
- MCP multiplies surfaces: Every MCP server is a tool contract plus a data path. Securing prompts while leaving MCP credentials over-scoped repeats the Sinch rollback pattern in a new protocol shape.
- Machine speed: A human analyst mis-clicking a dashboard is an incident. An agent looping across tools is a batch incident. Governance that was “good enough” for 50 human queries a day breaks at 5,000 autonomous ones.
Wave A vendors are racing to add tool and identity context — rightly so. Wave B vendors already sit on the enforcement plane data teams fought for through GDPR, HIPAA, and SOX programmes. The Production Perimeter is that plane extended to non-human principals.
Why data-governance DNA is the structural advantage
A greenfield AI-security startup starts from the prompt. A data-governance company starts from who may see which cell, on which engine, under which audit trail. For agents, the second starting point is the expensive one to build and the cheap one to already have.
Trust3 AI is one current face of that lineage — the same organisation previously known as Privacera, which spent years on cross-engine fine-grained access before “agentic” was a category. We cite it because the architecture it documents is the architecture production agents need, not because of any reseller or partner relationship. We describe a documented pattern; verify it against Unity Catalog, Snowflake, Ranger, and Lake Formation on your own estate.
Three structural reasons this DNA matters now:
1. Cross-engine policy was the hard problem before agents existed. Multi-cloud estates already needed the same logical rule in Trino, Snowflake, Databricks, and a lake. Ranger plugins intercept the query inside the engine; policy-sync approaches translate the rule into native constructs (masking UDFs, Unity Catalog row filters, Lake Formation data filters). Agents did not invent that split. They increase the number of principals hitting those engines at once. A company that already ships that translation layer is not learning FGAC on the customer’s critical path.
2. Observability and authorization have to meet in one control plane. That platform’s public positioning — discover every agent, observe every decision, secure every action across Copilot, Bedrock, Databricks Agent Bricks, MCP, and custom builds — is explicit that traces and permits must unify at the moment of action, not in separate SKUs. Forrester’s December 2025 agent control-plane framing (quoted on that site) describes an enforcement layer outside the build plane. Whether or not you buy any product, that framing is the right test for your own stack: does policy sit between the agent and the data, or only in a dashboard afterwards?
3. Purpose beats job title for machines. Human RBAC (“analyst”, “support tier 2”) maps poorly to ephemeral agent identities with a declared task. Purpose-based access — what this run may do for this job, time-boxed — is how you avoid standing admin keys on service accounts. JIT grants and auto-expiring scopes are the same idea native engines and PBAC vocabulary already point at. Startups that only score prompts still have to invent identity, purpose, and expiry. Data-access platforms already live there.
What we are not claiming: that any single vendor is mandatory, that prompt security is solved, or that mid-market operators must buy an enterprise control plane on day one (see our Governance Poverty Line). We are claiming that teams who spent years on FGAC, catalog, and audit trails have a faster path to a defensible Production Perimeter than teams starting from a prompt firewall alone. That is the advantage. Everything else is packaging.
Practical playbook — faster to prod and to the right data
Each step has a definition of done. Skip one and you are building a demo, not an operating commitment.
The sequencing trick: do not serialise “govern then ship.” Shrink the data surface of one workflow in parallel with the agent. A retrieval allowlist and a least-privilege principal for that workflow beat a twelve-month catalog programme that never meets the pilot.
1. Inventory agent identities like human roles
Done when: Every production agent has a named service principal, an owner, a declared purpose, and no shared admin credentials. Shadow AI counts — browser copilots with SSO to your warehouse are agents your perimeter does not see.
Acceptance criteria: you can list agent identities in one inventory; each maps to an approver; no credential serves both batch ETL and customer-facing tools.
Time-to-prod shortcut: clone the human role that already does the job, then strip write and export. Do not invent a new “AI admin” role.
2. Contract every tool (MCP, SQL, API)
Done when: Each tool documents allowed operations, field allowlists, row filters, rate limits, and whether writes are permitted. “Read CRM” is not a contract; “read contacts.email, contacts.tier for account_id in caller scope; no export” is.
OpenAI’s practical agent guide recommends maximising a single agent’s toolset only until overlap causes mis-selection — tool design is scoping.
Time-to-prod shortcut: one read tool and one draft tool before any write tool. MCP does not change that order.
3. Put enforcement in-path, not beside it
Done when: A policy decision (allow, deny, mask, scope) executes before data returns to the model context. Gateways (Portkey, Kong, LiteLLM with caveats), engine plugins (Ranger, Unity Catalog), Lake Formation filters, or a unified trust layer — pick the seam your architecture already trusts.
Log-only sidecars remain valuable for debugging; they are not a perimeter — that is the red path in the perimeter diagram above.
Time-to-prod shortcut: if the table already has a human row filter, bind the agent principal to that filter before you write a new policy language.
4. Tag lineage on RAG corpora and live queries
Done when: For each production answer you can point to source objects: document IDs, table names, chunk versions. Catalog entries link retrieval indices to upstream classification. If retrieval returns the wrong chunk, a stronger model still lies confidently — our AI-ready data post covers retrieval quality; lineage covers accountability.
Time-to-right-data shortcut: an allowlist corpus for the workflow (named folders, tagged tables) is faster than “embed the share drive.” Wrong data in the index is a perimeter failure that looks like hallucination.
5. Ship evals before scale — built from pilot failures
Done when: A versioned eval set covers happy path, refusal cases, and known failure modes from pilot (misclassification, over-broad tool call, PII near-miss). LangChain’s 2026 State of Agent Engineering (n=1,340) reports 89% observability adoption but only 52% running systematic evaluations — the gap that turns incidents into repeat incidents.
Time-to-prod shortcut: ten eval cases from real tickets beat a hundred synthetic ones. Include two “must not see” cases (other tenant, masked column).
6. Monitor agent drift against baseline
Done when: Alerts fire on material deviation from shipped behaviour: new tables in tool traces, rising tool-call cardinality, permission changes on service accounts, retrieval sources shifting off-catalog. Drift is not model version change alone — it is scope creep in production.
Time-to-prod shortcut: log tables_touched and tool_name from day one. You can add pretty dashboards later; you cannot reconstruct an unlogged first week.
The loop only closes if the comparison is automatic — baseline on one side, live traces on the other, a named owner for the difference:
Scroll sideways on a narrow screen to see the full loop.
7. Human gate on irreversible actions
Done when: External send, financial write, delete, and production config change pause until approval — not as optional UX polish. The Replit production-database deletion and Air Canada chatbot liability curriculum applies regardless of perimeter maturity.
Time-to-prod shortcut: ship read-and-draft to production with a human send. Autonomy is a later release, not a launch requirement.
Put the seven steps together and the shape is the one that keeps showing up in agents that survive their first quarter:
Scroll sideways on a narrow screen to see the surviving-agent shape.
Glossary — terms your auditor (and your eval set) will use
Dense definitions for security reviews, procurement questionnaires, and answer engines. Where we maintain a standalone page, we link it.
Production Perimeter — Enforced boundary where agent tools meet data; policy runs in-path before rows or chunks enter model context.
Time to right data — How quickly an agent can be bound to the tables, rows, and documents it is allowed to use. Distinct from time to first demo. Production speed is mostly this, not model choice.
FGAC (fine-grained access control) — Access restrictions below table level: rows, columns, masks, encryption. Documented natively across engines — see Unity Catalog, Snowflake column-level security, and Lake Formation data filters.
OLAC (object-level access control) — File, bucket, or object-store permissions; necessary where SQL FGAC does not block direct object-store bypass.
RBAC / ABAC / TBAC — Role-based, attribute-based, and tag-based access. Humans often start at RBAC; agents usually need attributes or tags (tenant, purpose, data class) because they do not have job titles.
PBAC (purpose-based access control) — Access decided from declared task purpose, not static job title. See /en/glossary/pbac/.
Lineage — Provable graph from source data through transforms to the asset an agent read or quoted. Catalogs automate this when agents use governed connections.
Data catalog — Inventory of datasets, owners, classifications, and (ideally) policies. Without it, every MCP server becomes a private undocumented warehouse.
Classification / tagging — Labels such as PII, PCI, tenant, jurisdiction. FGAC and retrieval allowlists consume these tags; they are not decoration.
Masking — Transforming a column (hash, last-four, null) so a principal can query a table without receiving sensitive values in clear text.
Agent observability — Traces of prompts, tool calls, data scopes, latency, cost, and outcomes. See /en/glossary/agent-observability/. Observability tells you what happened; it does not alone tell you whether it was allowed.
Agent drift — Production behaviour or data-scope deviation from baseline evals and policy: new data sources, tools, or permissions without a controlled release.
Runtime policy enforcement — Deny, mask, or scope before the action completes. Contrasts with log-only or batch-recertification models.
RAG vs tool access — Retrieval over a corpus vs live queries/API calls. Different risk profiles: stale or over-broad chunks vs over-privileged SQL. Both need perimeter controls; neither replaces the other.
Retrieval allowlist — The named set of documents or tables an agent may embed or query. The fastest FGAC for unstructured data.
MCP (Model Context Protocol) — Standard seam for tools and context. Govern like any API: identity, scope, logging, rate limits. See /en/glossary/mcp/.
Evals — Structured tests with pass/fail criteria on agent behaviour. See /en/glossary/evaluation-evals/. Required for ship gates and drift comparison.
HITL (human-in-the-loop) — Human approval on consequential steps. See /en/glossary/human-in-the-loop/.
JIT grants — Time-boxed elevation for a declared purpose; avoids standing write access on agent service accounts.
DSPM (data security posture management) — Discovery and classification of sensitive data across stores — including shadow datasets agents might find first.
Policy sync — Same logical rule enforced across engines (e.g. Snowflake masking + Unity Catalog row filter + Lake Formation filter) without manual duplication per platform.
Service account sprawl — Shared, over-privileged, or untracked credentials agents inherit; primary silent cause of “wrong data” incidents.
Agent identity — The non-human principal the engine sees (service account, workload identity, delegated user). If it is borrowed from a human or shared across agents, you have already lost the audit.
Shadow AI — Ungoverned copilots, plugins, or personal API keys operating outside inventory and perimeter.
Prompt injection / tool abuse — Attacker- or document-controlled instructions that change agent behaviour, including calling tools the user did not intend. Wave A’s home turf; still requires Wave B if the tool can return the wrong rows.
Guardrails — Input, tool, and output constraints. See /en/glossary/guardrails/. Necessary; not a substitute for FGAC.
Who should do what
No agents in production yet. Do not buy a platform. Map one workflow, bind a least-privilege identity to existing human row filters, contract one read tool, add two “must not see” evals, keep send human. Read our Governance Poverty Line piece if budget pressure tempts you to skip inventory.
Pilot stuck in “IT review”. You likely lack perimeter evidence, not model capability. Produce a one-page diagram: identities, tools, enforcement points, sample trace, eval results including a denied row.
Rolled back a live agent. Treat rollback as curriculum. Re-scope to one workflow; add in-path FGAC on the data store the agent actually hit; rebuild evals from the failure cases; re-pilot with human gates unchanged.
Scaling past one workflow. Standardise gateway or trust-layer patterns, shared catalog, drift dashboards, and JIT patterns. Now enterprise platform RFPs may match reality — not before.
Predictions (our read)
- Agent-security startups will acquire or deeply partner into data FGAC — prompt-only TAM is too thin for production rollbacks driven by PII exposure.
- MCP will force gateway consolidation — too many unaudited tool servers for CISOs to accept; enrichment of gateway logs with agent identity and purpose becomes default.
- Buyers will ask “show the perimeter” before “show the eval score” — because Sinch-style rollbacks are data incidents first.
- Native catalog AI-governance features (Databricks AI Gateway, Snowflake policies, Lake Formation filters) absorb mid-market share below the Governance Poverty Line before standalone platforms do.
- Drift monitoring becomes a compliance artefact — EU AI Act and sector rules increasingly expect ongoing behaviour evidence, not a one-time red-team PDF.
Quiet close
We build AI Operating Layer engagements around the same perimeter discipline: narrow scope, contracted tools, enforcement at the data seam, evals, traces, human gates. That sits across our AI governance and data governance practices — one governed system, not a chatbot plus a spreadsheet of policies.
If you want a straight read on where your first agent crosses the Production Perimeter — and what to enforce before scale — start with a Web & eCommerce quote or US → EMEA.
Sources
- Sinch AI Production Paradox — press release and production challenges chapter
- Trust3 AI — platform overview
- Privacera — About FGAC (secondary; pattern reference)
- Databricks — Unity Catalog overview
- Snowflake — Column-level security intro
- AWS — Lake Formation data filtering
- Apache Ranger — https://ranger.apache.org/
- Lakera — AI security platform
- Preamble — Agent security assessments
- Guardrails AI — https://guardrailsai.com/
- NVIDIA — NeMo Guardrails
- LangChain — State of Agent Engineering 2026
- OpenAI — Practical guide to building agents (PDF)
- Yerbabuena — Why agentic AI projects fail, AI-ready data, Governance Poverty Line
Frequently asked questions
What is the Production Perimeter?
Our term for the enforced boundary where agent tools — MCP servers, SQL connectors, APIs — meet governed data. Policy, masking, and row filters must run in-path before a query or retrieval returns rows. Logging alone is not a perimeter.
Why do data governance companies have an advantage for agentic AI?
Agents amplify existing access paths at machine speed. A company that already enforces row and column policy, syncs that policy across engines, and can reconstruct lineage already owns the hard half of agent production. Prompt firewalls do not row-filter a SQL tool or tell you which chunks grounded an answer.
What is agent drift?
Behaviour or data-scope deviation from your baseline evals in production — new tables queried, tool mix shifting, permission creep on a service account. Detect it by comparing live traces to the eval set and policy inventory you shipped with, not by model version alone.
How do we get agents to production faster without exposing the wrong data?
Shrink the data surface first, then the agent. One workflow, one identity, contracted tools, in-path FGAC on the tables that workflow actually needs, evals from day one, a human gate on irreversible actions. A full catalog programme is not a prerequisite; a retrieval allowlist and least-privilege principal for that workflow are.