Multi-tenant AI in SaaS: the 4 leak paths row-level security misses

Adding AI to a multi-tenant SaaS product is not a feature problem. It is a tenancy problem, and most teams find that out in the worst possible place: an enterprise security review, six weeks after the feature already shipped.

Your tenant boundary was built for queries. A tenant_id column, row-level security, maybe a schema per customer. It is well tested, it has held for years, and it covers almost nothing about what happens to those rows after they leave the database. The moment your code assembles a prompt, customer data crosses into a system that has no concept of a tenant at all.

This is written for founders, CTOs, and engineering leads at B2B SaaS companies that already have paying customers and are starting to sell upmarket. If you are building an internal tool or a single-customer AI wrapper, none of this applies. If your product holds data for 50 or 5,000 separate organisations and you are about to put a model in front of it, the next 10 minutes will save you a quarter of engineering time.

Here is what you will get: a plain definition of multi-tenant AI, the 4 specific paths where tenant data escapes once an LLM is involved, the isolation patterns worth choosing between, the 6 controls that hold up under an enterprise security questionnaire, an honest view of the cost, and a worked example with real numbers.

‍

Key takeaways

• Multi-tenant AI is the practice of serving LLM features to many customer organisations from shared infrastructure while keeping each tenant's data, context, and outputs strictly separated.

• Row-level security protects the query. It does not protect the prompt, the retrieval context, the tool call, or the trace, which are the 4 places tenant data actually leaks.

• Isolation enforced in application code is not isolation. One missing filter in one code path is a breach, and LLM code paths multiply fast.

• Choose your vector isolation pattern deliberately: silo (an index or namespace per tenant) buys a clean audit story, pool (shared index with filters) buys cost efficiency, and a hybrid serves most real products.

• Observability is the quietest leak. Traces, prompt caches, and eval datasets routinely copy raw tenant content into tools that have no tenant model.

• Sensitive information disclosure now ranks second in the OWASP Top 10 for LLM applications, which means your buyer's security team is already asking about it.

• The deadline that matters is not your release date, it is the first enterprise security questionnaire with an AI section on it. IT security review is the single biggest source of delay between a buyer picking you and actually signing.

‍

What is multi-tenant AI?

Multi-tenant AI is the practice of delivering AI features to many separate customer organisations from shared infrastructure, while guaranteeing that no tenant's data, retrieved context, model output, or usage record can reach another tenant. It is the same promise your database already makes, extended across a much wider surface.

The word "shared" is doing the work here. Almost nobody runs a dedicated model, vector database, and inference endpoint per customer, because the cost at 300 tenants is absurd. So you share an embedding pipeline, a vector store, a prompt template, a model endpoint, a cache, and an observability stack. Every one of those shared components is a place where tenant A's content and tenant B's content sit in the same process, the same index, or the same log line, separated only by a piece of logic somebody wrote.

That is a real difference from the multi-tenancy you already run. In a classic SaaS request, data flows out of the database into a response and back to one authenticated user. The path is short and one guard at the database covers it. In an AI request, data flows out of the database into a retrieval step, into a prompt, into a third-party model, into a response, into a trace, into a cache, and sometimes into an evaluation dataset that an engineer opens on their laptop three weeks later. The path is long, branching, and mostly invisible to the controls you already trust.

‍

Why row-level security stops at the prompt

Row-level security answers one question: which rows may this session read. It answers it deterministically, which is why it has been the backbone of SaaS tenancy for a decade. It stops working the moment that answer is handed to something else.

Think about what a typical AI feature does. It authenticates a user, resolves their tenant, pulls rows under a tenant filter, serialises them, concatenates them with a system prompt, sends the whole thing to a model, receives free text back, and renders it. Row-level security governs step 4 of 8. Steps 5 through 8 happen in application code, vendor SDKs, and third-party infrastructure, where the concept "tenant" does not exist unless you invented it yourself.

The failure mode is rarely dramatic. Nobody wakes up to find tenant A reading tenant B's database. What happens is smaller and much easier to ship: a retrieval helper written for one feature gets reused in another where the tenant filter is passed differently and silently defaults to none. A prompt template caches the "recent examples" block across requests to save tokens. A weekly summary job iterates over all tenants and builds context before it establishes tenant scope. A support engineer exports 200 traces to debug a hallucination and every one carries verbatim customer content.

None of those is a security incident in the movie sense. Each of them is a finding that stops a deal, and at least one of them is present in most AI features we have reviewed that were built quickly by a small team under pressure to ship.

There is a second, subtler problem. A model does not distinguish between instructions and data. If tenant B can get text into a place where tenant A's request will read it, that text can influence tenant A's output. Shared knowledge bases, shared example pools, and shared few-shot libraries all create this path. It is not a hypothetical: prompt injection and sensitive information disclosure are the top two entries in the OWASP Top 10 for LLM applications, and sensitive information disclosure climbed from sixth place to second in the 2025 revision precisely because teams kept shipping this pattern.

‍

The 4 leak paths that row-level security does not cover

Once you accept that the boundary has to move outward, the work becomes concrete. There are 4 places where tenant data leaves the protected zone in an AI feature. Map them, and the security questionnaire stops being a fishing expedition.

‍

1. The prompt

The prompt is the first place tenant data leaves your perimeter, and the one teams reason about least carefully, because it looks like string concatenation rather than a data transfer.

Everything you put in the prompt goes to a model provider: the system prompt, the user message, and every piece of context you injected. Your data protection story is now your provider's data protection story, and your buyer will ask you to prove it. That means a signed agreement covering training use, a documented retention window, a named region, and a subprocessor entry.

The tenant-specific risk sits in how prompts get built. Templates that pull "similar past cases" are the usual offender, because the most useful examples are almost always drawn from your whole corpus rather than one customer. A pattern that improves output quality by 10 percent can hand tenant A a sentence from tenant B. The fix is boring and effective: every piece of context carries an explicit tenant tag, and the prompt builder refuses to assemble a prompt containing more than one.

2. The retrieval context

Retrieval is the leak path with the highest blast radius, because a single wrong filter returns the most relevant content across every customer you have, which is exactly the content you least want to move.

Vector databases are not relational databases, and most do not have row-level security. Isolation is expressed as a namespace, an index, or a metadata filter you pass at query time, which means it lives in your query code rather than in the storage engine. That is a real downgrade from your relational tier, and worth saying out loud, because engineers who trust the database extend that trust to the vector store by instinct.

Two further details catch people. First, deletion. When a tenant deletes a record or offboards, the source row disappears and the embedding usually does not, so your AI feature keeps answering from data the customer believes is gone. Under GDPR that is a deletion request you failed. Second, provenance. If a chunk arrives without a tenant identifier attached at ingestion, no query-time filter can recover it, and you will re-index your whole corpus to fix it.

3. The tool call

The moment your AI feature stops summarising and starts acting, the tenant boundary has to extend into every tool the model can invoke, and the model is the one choosing the arguments.

This is where AI features become genuinely different from the rest of your product. An agent that can call get_customer, send_email, or update_record generates those arguments from text, and text can be influenced. A support ticket carrying an instruction, a PDF with a hidden line, or a CRM note written by an outside party can all steer a tool call. If your tools trust the arguments the model supplies, tenant scope is decided by a language model.

The control is simple to state and easy to skip. Tenant context comes from the authenticated session, never from the model. Every tool re-derives the tenant from the request context and re-authorises the target object against it, exactly as a well built API endpoint would. The model chooses which tool to call. It never chooses whose data.

4. The trace, the cache, and the eval set

Observability is the quietest leak path, and the one most likely to be missing from your architecture diagram, because a well meaning engineer added it while debugging a bad answer.

An LLM feature is hard to debug without seeing the actual prompt and output, so almost every team ships full prompt and completion capture to a tracing tool. That tool is usually a third-party SaaS product, outside your tenant model, often outside your data region, and readable by every engineer with a login. You have quietly built a second copy of your customers' most sensitive content somewhere with weaker controls than your database.

The same applies to two adjacent artefacts. Prompt and semantic caches key on content, so a cache that is not partitioned by tenant can serve one customer a response computed from another customer's data. Evaluation datasets are worse, because their whole purpose is to be durable, and a golden set built from production traffic becomes a permanent, version-controlled export of real customer records.

Treat all three as production data stores. Redact or tokenise before capture, partition caches by tenant, keep eval sets synthetic or contractually cleared, and put a retention window on traces that you can actually point to in a questionnaire.

‍

If you are scoping this work now, our SaaS founder's AI blueprint covers the architecture and sequencing decisions behind AI features in an existing product, including how to phase a build so security work is not the thing you discover last. It is free and takes about 20 minutes to read.

‍

Silo, pool, or hybrid: choosing an isolation pattern you can defend

The core architectural choice in multi-tenant AI is how you separate tenants in the retrieval layer, and there are exactly 3 credible answers. Pick one deliberately, write down why, and you have most of your security narrative already.

Silo means each tenant gets their own index, collection, or database instance. Nothing is shared, so a missing filter returns nothing rather than someone else's data. The audit story is one sentence long. The cost is operational: hundreds of indexes to provision, migrate, back up, and monitor, plus per-index minimums that make small tenants expensive. It is the right default when tenants are few, large, and regulated.

Pool means one shared index with a tenant identifier on every vector, filtered at query time. It is dramatically cheaper, it scales to thousands of tenants without operational pain, and it makes cross-tenant analytics possible. It also puts the entire boundary in your query code, which is the exact thing this article warns about. Pool is defensible, but only with a single enforced access layer that no feature can bypass, plus tests that assert isolation on every code path.

Hybrid is what most mature products end up with. Pool by default, silo for the tenants whose contracts or regulators demand it, with one interface in front of both so feature code does not know which it is talking to. It costs more once and saves you from repricing your architecture the first time a large customer asks for dedicated storage.

‍

The mistake to avoid is drifting into pool by accident. Plenty of teams pick a shared index because it was the default in a quickstart guide, never document the decision, and then cannot answer "how is tenant data separated in your vector store" without a week of code reading. The pattern matters less than being able to describe it.

‍

The 6 controls that hold up under review

These are the controls that consistently survive contact with an enterprise security team. They are not exotic, and none of them require a platform rewrite.

• One tenant-scoped data access layer. Every read that can reach a prompt goes through a single module that takes tenant context as a required argument and cannot be called without it. No feature gets its own retrieval helper.

• Tenant context from the session, never from the model. Derive it at the edge, carry it in the request context, and re-derive it inside every tool and every background job before any data is touched.

• Tenant tags at ingestion, not at query. Every chunk, embedding, and cache key carries a tenant identifier written when the record is created. Query-time filtering is the second line of defence, not the first.

• Isolation tests as a permanent suite. A test that seeds 2 tenants, runs every AI code path as tenant A, and asserts that no tenant B content appears anywhere in the prompt, the response, or the trace. Run it in CI and it becomes a real answer in a questionnaire.

• Redaction before observability. Traces capture structure, token counts, latency, and identifiers by default. Raw content capture is opt-in, time boxed, and logged.

• A deletion path that reaches the AI layer. When a record is deleted or a tenant offboards, embeddings, cached responses, and traces are removed on the same schedule as the source row, and you can demonstrate it.

‍

What multi-tenant AI actually costs, and where teams overbuild

The honest answer is that doing this properly adds somewhere between 3 and 6 weeks of engineering to a first AI feature in an existing multi-tenant product, and close to nothing to the second one. That asymmetry is the entire point.

Most of that is one-time platform work: the access layer, the tenant-aware retrieval interface, ingestion tagging, the isolation test harness, and redaction in observability. Build it once and every later AI feature inherits it. Skip it and you pay a smaller tax in every feature, forever, plus one large unplanned payment the first time a buyer's security team reads your architecture.

The running cost is more manageable than teams fear. Pooled vector storage at a few hundred tenants is rarely the dominant line item; inference is. What surprises people is per-tenant cost attribution. If you cannot say what one customer's AI usage costs, you cannot price the feature, spot the tenant whose 2 power users generate 40 percent of your token spend, or enforce a fair-use limit. Tag every model call with a tenant identifier from day one. It costs nothing at build time and is painful to retrofit.

Where teams overbuild is equally consistent. A dedicated model deployment per tenant that nobody asked for is the most expensive mistake available, and it solves a problem pooled inference with proper request isolation already solves. Per-tenant fine-tuning is the second, because it creates a permanent, hard to delete copy of customer data inside model weights and turns a deletion request into a retraining project. Neither is necessary for most B2B SaaS products. Reach for them when a contract requires it, not when a diagram looks tidier.

‍

Your real deadline is the first security questionnaire

Here is the commercial reframe that changes how teams sequence this work. The deadline for multi-tenant AI is not your release date. It is the first enterprise security questionnaire that comes back with an AI section on it, and that arrives right when the deal is closest to closing.

The pattern is painfully consistent. A mid-market or enterprise buyer picks you, procurement starts, and the questionnaire lands. It asks which model provider you use, whether prompts are used for training, where inference happens geographically, how tenant data is separated in retrieval, how long prompts and completions are retained, and whether the model can act on customer data. According to the G2 2026 Buyer Behavior Report, IT security review is the single biggest source of delay between a buyer selecting a vendor and completing the purchase, named by 39 percent of buyers overall and half of enterprise buyers. Those questions are not a formality. They are the gate.

A team that built the boundary deliberately answers in an afternoon, with a diagram, a test suite, and a subprocessor list. A team that shipped the feature first spends 3 to 6 weeks doing forensic archaeology on their own code while the champion inside the account loses momentum. Same product, same quality of AI feature, completely different quarter.

This is why we treat the questionnaire as a design input rather than a compliance chore. Write the answers before you write the feature. If you cannot answer "how is tenant data separated at every layer of this feature" in 3 sentences, the architecture is not finished yet.

‍

A worked example: adding AI summaries to a 400-tenant B2B SaaS

Take a concrete shape. A B2B operations platform, 400 customer organisations, Postgres with row-level security, roughly 40 million records, selling mid-market and starting to see enterprise deals. The team wants an AI summary on every project, generated from notes, documents, and activity history.

The naive build takes about 3 weeks. Pull the project and its related records, embed the documents into a shared vector index, retrieve the 20 best chunks at request time, build a prompt, call a model, cache the result by project ID, and send full traces to an observability tool. It works, it demos beautifully, and it contains at least 3 of the 4 leak paths above.

Retrieval filters on project ID rather than tenant ID, which is fine until 2 tenants have projects with colliding identifiers or a helper is reused without the filter. The cache is keyed on content, so 2 tenants with similar documents can share a cached response. Traces carry raw customer notes into a third-party tool with no retention policy. Nothing has gone wrong yet, and nothing will, until the day it does or until somebody asks.

The deliberate build takes about 6 weeks and differs in ways that are mostly unglamorous. Ingestion writes a tenant identifier onto every chunk. Retrieval goes through one interface that requires tenant context and filters on tenant first, project second. The cache key includes the tenant identifier. Tool calls, once summaries become an agent that can update records, re-derive tenant scope from the session. Traces capture token counts and identifiers by default, with a 7 day window and content capture behind a flag. An isolation test seeds 2 tenants and asserts that a request as tenant A never surfaces tenant B content.

Three extra weeks. In exchange, the second AI feature ships in a week instead of three, the security questionnaire is answered from documentation rather than from a code audit, and a large customer asking for dedicated storage becomes a configuration change rather than a rebuild. That trade is not close.

‍

How we approach multi-tenant AI builds

When we take on AI work inside an existing SaaS product, the first artefact is not a prototype. It is a one-page data flow tracing exactly where tenant data travels, from the database through retrieval, prompt, model, response, cache, and trace, with the boundary drawn on it. It takes half a day and consistently finds 1 or 2 paths the team had not considered.

From there the sequence is platform first, feature second: the tenant-scoped access layer and ingestion tagging go in before any model call is written, because retrofitting them means reprocessing your corpus. We write the security questionnaire answers alongside the architecture, which is the cheapest way we know to keep a build honest.

That approach comes out of building production platforms rather than demos. If you want a second pair of eyes on a build that is already underway, our AI development team reviews existing AI architectures as a fixed-scope engagement, and our SaaS development work covers the platform layer underneath it.

‍

Closing the loop

Multi-tenant AI comes down to one idea: your tenant boundary has to move outward to cover the prompt, the retrieval context, the tool call, and the trace, because the database guard you already trust stops at the query. Do that work up front and AI features become a normal part of your roadmap. Skip it and every enterprise deal pays for it in delay.

The goal is not a smarter product. It is a product where adding an AI feature does not trigger a fire drill, where the security questionnaire is answered from a document you already wrote, and where the architecture holds when a customer 10 times larger than your biggest asks how it works.

If you want the wider architecture and sequencing view, the SaaS founder's AI blueprint is a free download that covers how to phase AI into an existing product without rebuilding it.

And if you are about to add AI to a product that already holds other people's data, it is worth an hour with someone who has drawn this boundary before. You can book a free call with our team and we will walk your data flow with you.