RAG assistant inside a SaaS product

How a RAG assistant inside a SaaS product changes your tenant model

If you are adding a document assistant for paying customers, every customer is a tenant, and a document leaking from one tenant into another's answers is a product failure that customers will treat as a breach, not an internal incident. The platform choice depends on whether you can legally and technically run multiple tenants on it, and tenant isolation has to be designed before the first document is ingested. Retrofitting isolation after embeddings already sit in a shared index is far more expensive than deciding the boundary up front. DeployOpen would operate the stack in your own cloud account or data center, so customer documents, embeddings and logs stay under your control. In that model you, the SaaS provider, become the processor for your customers' documents, which means your customer contracts need to reflect processing on documented instructions and the security measures that GDPR Article 28(3) and Article 32 expect. Confirm with your legal team how that flows down to your sub-processors and hosting arrangements. The short answer on platforms is that the license of the chosen application framework decides whether it can serve customers at all. Dify's Dify Open Source License, a modified Apache License 2.0, forbids using the source code to operate a multi-tenant environment, where one tenant is one workspace, without written authorization, so a multi-customer product built on Dify needs a commercial agreement before launch. Open WebUI's license restricts changing its branding beyond small deployments, and that matters if customers see the interface. Check the current license text of every component, including the vector database, against your planned tenancy model before you pick the framework.

Requirements a customer-facing RAG assistant must meet

This table lists the requirements a document assistant must meet once customers, not just staff, use it inside a SaaS product, and what each one means for the build. Read each row as a design decision to settle before the first tenant is onboarded, since several of them constrain the platform choice itself. The main takeaway is that licensing and tenant isolation are decided early, and retrofitting them later is costly.

Requirement	Where it comes from	What it means for the build
Tenant isolation of documents, chunks, embeddings and chat history	Customer contracts and GDPR security of processing; Qdrant access controls	Give each tenant its own collection or namespace, enforce the tenant filter in the retrieval layer as well as the API, and test that one tenant's query never returns another tenant's chunks. Confirm whether chat history and cached answers are scoped per tenant too.
License permission for multi-tenant operation	Dify license forbids operating a multi-tenant environment from the source without written authorization; Open WebUI License clause 4 limits branding changes to small deployments or with written permission	Dify is not a fit for a multi-tenant SaaS offering without a commercial agreement, and Open WebUI branding can only be altered under its listed exceptions. Have counsel review the exact license text against your product model before choosing a platform, and consider Apache-2.0 options such as RAGFlow where that fits.
Per-customer metering and budgets	Model and embedding usage is billed per call; gateway features in LiteLLM include cost tracking	Attribute every embedding, retrieval and generation call to a tenant ID, set spend limits per plan, and decide what happens at the limit. Confirm that your gateway can report cost per tenant before promising usage-based pricing.
Answer evaluation before each release	Internal quality expectations and customer-facing accuracy claims	Maintain a test set per product area with expected source documents, run retrieval and answer checks on every change to prompts, chunking or models, and block release on regressions. Record results so that a release can be traced back to its evaluation run.
Audit logs of authenticated access	Qdrant JSON audit logging of authenticated API operations since v1.17.0; EU AI Act Articles 12(1) and 19(1) log duties for high-risk systems	Log who queried which collection and which documents were returned, and retain logs for the period your contracts and any high-risk classification require. Article 19(1) sets at least six months unless other law provides otherwise. Confirm with counsel whether your use is high-risk, since that determines whether these duties apply.
GDPR Article 28(3) processor instructions for customer data	GDPR Article 28(3) requires documented instructions and Article 32 security measures from processors	Your customer agreements must define the processing you perform on tenant documents, including subprocessors such as hosted model APIs. Document the instructions, limit processing to them, and confirm which transfers, if any, occur when embeddings or prompts reach external model providers.
Data Act switching rights for data processing services	EU Data Act Articles 23, 25 and 50, with switching charges prohibited from 12 January 2027 under Article 29	Customers must be able to move to another provider or on-premises infrastructure with a maximum two-month notice period and 30-day transition. Plan exportable formats for documents, embeddings and metadata, and confirm with counsel whether your service falls under the Data Act definition of a data processing service.

Which RAG platform fits a SaaS product and why Dify's license matters

A document assistant offered to customers depends on three things: whether the platform's license allows multi-tenant operation, whether it isolates each customer's documents and embeddings, and whether the vector store can enforce access per collection. The cards below compare the candidates on those points.

Dify: check the license before choosing

Dify's Open Source License forbids operating a multi-tenant environment from the source code without written authorization, where one tenant is one workspace. The self-hosted Community edition is limited to a single workspace. The Enterprise edition is custom priced and adds multiple workspaces and a commercial license. A SaaS team serving customers as separate tenants should get a written answer from the licensor or an Enterprise quote before selecting Dify. The current release noted in the supplied facts is 1.17.1.

Open WebUI: branding clause limits customer use

Open WebUI is distributed under the Open WebUI License, which is BSD-style but adds a branding rule in clause 4. Altering or removing Open WebUI branding is allowed only for deployments with no more than 50 end users in any rolling 30 day period, with written permission, or under other listed exceptions. A customer-facing assistant with a broader audience would likely need written permission for any white-labeling. Confirm the exact terms with the license holder before building a branded product on it.

RAGFlow: document parsing for ingestion

RAGFlow is an open source retrieval-augmented generation engine under Apache-2.0, with strengths in document parsing and chunking that matter when customers upload varied files. Its licence permits commercial embedding in a product. Confirm how the version you plan to run separates tenants: knowledge bases, user accounts and retrieval filters must keep one customer's documents out of another customer's answers. Test this with two synthetic tenants before release.

AnythingLLM: workspace-based isolation

AnythingLLM is licensed under MIT and organizes documents and chat into workspaces, which maps naturally to a per-customer boundary. The MIT license places no branding or multi-tenant restriction on the code. The team must still confirm the tenant isolation model in the version it runs, including whether workspace boundaries also apply to embeddings, API keys and administrative roles, and whether a customer administrator can see another workspace's files.

Qdrant: per-collection permissions

Qdrant is an open-source vector search engine under Apache-2.0. Since v1.9.0 it supports JWT-based access control with read or write permissions per collection, so one collection per customer can be enforced at the database layer. By default, self-deployed Qdrant instances listen on all network interfaces and have no authentication configured, so the deployment must enable API keys, TLS and access rules before any customer data is loaded. The latest release in the supplied facts is v1.19.2.

Choosing how to isolate each customer's embeddings

All customers' vectors sit in one collection, and every query carries a tenant identifier that the application must apply as a filter before results return. This is the lowest-cost model because one index, one set of shards and one backup job serve every customer. Isolation depends entirely on the application code path, so the filter is a control you have to prove, not a property the database provides on its own.

Enforcement: The tenant key is stored as a payload field on every point, and the query layer adds it as a mandatory condition. Nothing in the database stops a query that omits it.
Failure mode: A single code path, background job or admin tool that skips the filter returns another customer's chunks to the wrong user. The failure is silent unless tests look for it.
Storage and replication: One collection stores each vector once per replica. At a Qdrant replication factor of 2, storage doubles, but this cost is shared across all tenants.
Operational load: One upgrade, one backup and one monitoring setup cover everyone, which keeps staffing light.
Typical fit: Customers with standard sensitivity who accept logical separation and whose contracts do not require dedicated infrastructure.

Before relying on this model, confirm that Qdrant API keys or JWT permissions are scoped as designed and run cross-tenant retrieval attempts against every query path.

Per-customer cost control and evaluating answers before release

Every tenant's usage should be metered from the first request, because a shared assistant hides cost until one customer's document batch or heavy chat usage arrives on the invoice. In a SaaS product, the metered units are model tokens, embedding jobs, re-indexing runs and vector storage per tenant. Route all model calls through an AI gateway such as LiteLLM, which provides cost tracking and budgets, and tag every request with a tenant identifier so spend can be attributed and capped per customer. Trace prompts, retrieved chunks, and answers in an observability tool such as Langfuse, which is MIT licensed except its ee directories. Confirm with the vendor which directories a given release includes before you distribute any of its code inside your product.

Check license fit before you add Arize Phoenix to the stack. Phoenix is licensed under the Elastic License 2.0, which prohibits providing the software to third parties as a hosted or managed service. A tracing tool that your customers reach indirectly through your product may fall under that restriction, so have counsel review the exact use before deployment. Decide the tooling in the architecture phase, because switching trace stores later means migrating historical records that may carry customer data and must follow your retention rules.

Make the evaluation gate a release blocker, not a report. Build a per-tenant test set drawn from each customer's own documents, plus a cross-tenant set that checks no answer for one customer cites another customer's files. Each release should run regression tests covering retrieval hits, citation accuracy, refusal behavior when the source is absent, and prompt-injection attempts embedded in uploaded documents. Agree sign-off criteria in writing with the product owner and the customer-facing contract team before the first run, including the thresholds that stop a release. The work and cost depend on the number of tenants, the document volume per tenant and the size of the evaluation set, so ask any implementation partner to price those three drivers explicitly, along with the effort to maintain the test sets as documents change.

How a SaaS RAG assistant moves from design to paid release

The sequence turns a document assistant idea into a customer-facing feature that is licensed, isolated per tenant, tested against agreed answer quality, metered and operated under a documented runbook. Each step closes with a written output that the next step depends on.

Review the platform license for multi-tenant use
Legal counsel and the product owner review the chosen platform's license before any build work starts. For Dify, the review covers the Dify Open Source License, which forbids using the source code to operate a multi-tenant environment, where one tenant is one workspace, without written authorization, and which also restricts removing or modifying logos and copyright information in the console. For Open WebUI, the review covers clause 4 of its license on branding. The vendor's written answers on multi-tenant use and branding go into a decision record, which states whether the platform, a commercial license or a different component is used. The step is done when that decision record is signed by legal and engineering leads.
Design the tenant model and isolation
Architects define how each customer's documents, embeddings, metadata and query logs are separated, whether by collection, index, database schema or deployment, and how the application enforces tenant identifiers on every retrieval call. The design includes key management for each tenant's data, the network path between the application and the vector store, and an admin access model for DeployOpen and the client's staff. Qdrant, for example, is not secure by default and needs authentication, TLS and per-collection access rules configured before any tenant data is loaded. The output is an isolation specification and a threat model covering cross-tenant retrieval, prompt injection through uploaded documents and leakage through logs. The step closes when the security lead accepts both documents.
Build tenant-scoped ingestion with permission and deletion handling
Engineers build the ingestion pipeline that parses customer files, chunks them, creates embeddings and writes them to the tenant's own collection. Each chunk carries the tenant identifier and any source-level access attributes, so retrieval can filter on them. The pipeline also defines how a customer deletion request removes source files, chunks, vectors and cached answers together, and how re-indexing runs when a document changes. Tests include a cross-tenant retrieval attempt, a deletion check that searches for removed content, and a malicious document containing instructions. The output is a tested pipeline with test records kept for each release. The step is complete when every test passes in a staging environment that mirrors production.
Set the evaluation set and release gate
Product and domain experts write a set of representative questions for the product's documents, with expected sources and expected answer content. Engineers run that set against the assistant to record baseline scores for retrieval hit rate, citation accuracy, groundedness and refusal behavior when the answer is not in the documents. Sign-off criteria define the minimum scores, the categories of question that must pass in full, and the owner who approves a release. Every model, embedding or platform upgrade reruns the set before release. The output is scored baselines and written sign-off criteria. The gate is closed only when the product owner and the security lead sign the criteria.
Meter usage, set budgets and log audit events
Engineers add per-tenant metering for model tokens, embedding calls, storage and query volume, and attach budgets that slow or stop a tenant's usage at agreed thresholds. A gateway or logging layer records who asked what, which sources were cited and which model answered, with retention set to meet the client's own policy and any applicable law. Cost control depends on the chosen model hosting, so the team confirms whether inference is paid per token to a provider or runs on the client's own servers. The output is a usage report per tenant that finance and support can read. The step is done when a test tenant reaches its budget limit and the report matches the logs.
Pilot with design partners, then release with a runbook
A small group of design-partner customers uses the assistant under written pilot terms, and the team reviews retrieval failures, citation complaints and cost data before general availability. The release plan requires regression testing against the evaluation set for every platform upgrade, because Dify shipped minor releases roughly monthly, including 1.16.0 on July 17, 2026, 1.17.0 on August 25, 2026 and 1.17.1 on September 10, 2026. The operating runbook covers upgrades, scheduled backups of the metadata database, file storage and vector store, and restore tests run on a schedule with recorded results. The output is a general release backed by a runbook that the operations team has rehearsed. Confirm backup frequency and retention targets with the client before the runbook is final.

Evidence customers, auditors and regulators will ask for

This table maps each piece of evidence a SaaS vendor shipping a document assistant will be asked for to who asks for it and which component of the stack produces it. Read it as a checklist to design logging, testing and contract records before the first enterprise customer asks, not after. The main takeaway is that most of the evidence comes from the product's own tenancy, logging and release process, and the auditor or regulator still decides what is sufficient.

Evidence	Who asks	Component that produces it
SOC 2 Type 1 or Type 2 inputs mapped to CC1 to CC9	Customers' security reviews, SOC 2 auditor	Control matrix, access reviews, change records and monitoring for the assistant. The auditor issues the report; DeployOpen only prepares controls and evidence. Report use is generally restricted.
Tenant isolation test results	Enterprise customers, security architects, pen testers	Automated cross-tenant retrieval tests and results from security testing of the assistant API
Qdrant audit logs and API key records	SOC 2 auditor, customer security teams, incident responders	Qdrant JSON audit logging (v1.17.0 and later), admin and read-only key inventory, per-collection JWT permissions
Per-tenant usage and cost reports	Finance, customers on usage billing, product managers	LiteLLM cost tracking or equivalent gateway metering, keyed by tenant and model
Evaluation reports for each release	Release approvers, customers asking about answer quality, compliance leads	Langfuse or Phoenix evaluation runs against a fixed test set, stored with the release version
GDPR Article 28(3) processor instructions and sub-processor records	Customers' privacy teams, data protection officers	Signed data processing terms, documented processing instructions, list of hosting and model sub-processors
Data Act switching and export test records	Customers planning exit, procurement, regulators	Export and migration test logs for collections, embeddings and source metadata, plus the notice and transition procedure
AI Act Articles 12 and 19 logs, six month minimum retention	Customers in regulated sectors, deployers, AI Act authorities	Event logs of queries, retrieved sources and outputs; confirm with counsel whether the feature is high-risk before setting retention

Questions to ask an implementation partner before building a customer-facing assistant

Start with cross-tenant isolation, because it is the failure customers will judge you on. Ask the partner to show the test suite that proves one tenant cannot retrieve another tenant's chunks, including adversarial queries that try to pull neighboring documents through similar wording, and ask how those tests run again after every re-index and every Qdrant upgrade. Ask which design they propose for tenant separation (one collection per tenant, payload-level filters enforced server-side, or separate clusters), and how Qdrant JWT tokens with per-collection read permissions, available since v1.9.0, are scoped and rotated. Ask whether the default Qdrant install is hardened before any tenant data arrives, since the security documentation states that self-deployed instances listen on all interfaces with no authentication by default. Ask for the reference to the license position too: confirm in writing whether the platform they propose, including Dify, may run as a multi-tenant service in your product, and in which version. The Dify license requires written authorization for multi-tenant use, so ask for the authorization or a commercial license quote before the architecture is fixed, not after.

Next, ask how a deletion request is carried out. A customer who asks for erasure expects the source file, every chunk and every embedding derived from it to disappear, and the partner should describe the mechanism that finds all three by document ID, the deletion job, the log of when it ran and how backups and snapshots are handled. For capacity, ask how many Qdrant nodes they plan for production and why. The Qdrant distributed deployment documentation recommends at least three nodes so quorum survives a node failure, and a replication factor of 2 doubles storage. Ask for the shard count, which should be a multiple of the node count, and for the storage math per tenant, since block storage with SSD or NVMe is required and network file systems and object storage are not supported. Ask how evaluations run before each release: which golden question set covers each customer's content, what answers must cite a source, what failure rate blocks a release, and who signs off. Confirm that evaluation sets contain no live customer data unless the contract allows it.

Finally, ask how the partner will supply SOC 2 control evidence. A SOC 2 report is issued by an independent CPA firm, so the partner should commit to producing evidence (access reviews, change records, backup restore tests, audit log exports such as Qdrant's JSON audit logging from v1.17.0) mapped to the Trust Services Criteria common criteria, and should never describe the deployment as certified or compliant. Ask which evidence is generated by the platform and which by your own processes. Last, ask what happens to customer data at exit. Under the EU Data Act, switching rules apply to data processing services, and from 12 January 2027 switching charges cannot be imposed, so ask for the export format for vectors, metadata and source documents, the time needed to move to another provider or on-premises infrastructure, and proof that the deletion of the old copy is performed and documented.

What a SaaS RAG assistant costs and what it needs from your team

How much does a SaaS RAG assistant cost to build and run?

No single figure applies, because cost depends on the components you choose and how many tenants you serve. The main cost lines are software licenses or Enterprise quotes, infrastructure for the application, vector database and workers, replication of the vector store, embedding and model inference, evaluation work, and ongoing maintenance. Open-source components such as Qdrant (Apache-2.0) carry no license fee for self-hosting, but the infrastructure, operations and specialist time still cost money. Some platforms, such as Dify Enterprise, are custom priced and require a written quote. Ask any partner for a quote that separates one-time build work from recurring infrastructure, licensing, model usage and support, and that shows how each line changes with tenant count and document volume.

What drives the cost of a multi-tenant RAG assistant up or down?

Tenant count, document volume and query volume drive most of the cost. Each tenant adds index storage, embedding work for its documents and inference calls for its questions. Replication also multiplies storage: according to the Qdrant distributed deployment docs, a replication factor of 2 doubles storage, and production clusters are recommended to run at least three nodes so quorum survives a node failure. Evaluation depth is another lever, since a larger test set and more frequent regression runs add effort. Per-customer cost control, such as usage limits and token budgets, keeps one heavy tenant from changing the economics for everyone else.

How long does it take to add a document assistant to a product?

The timing depends on decisions made early, not only on build effort. The main factors are the license decision, the tenant isolation design, the volume and messiness of customer documents, and how deep the answer evaluation needs to be. A license question that is still open can stop the project, so resolve it first. Isolation design requires review by your architects and security team before any indexing starts. Ask a partner for a timeline tied to a written scope with named deliverables, and ask which of those phases depend on your team's availability for reviews and test data.

What does the work need from our team?

A project of this kind needs four groups of people. Product owners must define which document types, customer workflows and answer behaviors are in scope. Identity and data owners must explain how tenants, users and document permissions are modeled in your systems. Your security team must review tenant isolation, secrets handling, network exposure and logging before release. Finally, you must provide sample tenant data for testing, or synthetic data that matches your real document shapes. Without that test data, evaluation results will not show how the assistant behaves for your customers.

Can our engineering team build this without an implementation partner?

Yes, a capable team with strong platform engineering and security skills can build a SaaS RAG assistant on open-source components. The areas that usually need specialist time are cross-tenant testing, which checks that one customer's embeddings and citations never surface for another, and evidence preparation for compliance reviews. Testing cross-tenant leakage requires deliberate adversarial cases, and compliance evidence such as access logs, configuration baselines and change records must be collected in a form an auditor can use. Estimate that work separately from the build, and confirm with your team whether it has the capacity for it.

Can we use Dify to serve customers in our SaaS product?

Only with written authorization or an Enterprise agreement, and this should be confirmed with the Dify licensor before any design work depends on it. The Dify Open Source License is a modified Apache License 2.0 that forbids using the source code to operate a multi-tenant environment, where one tenant is one workspace, without written authorization. The self-hosted Community edition is limited to a single workspace, so a product serving many customers will need a different license path. Check the Dify LICENSE text with your legal team, and do not assume the Enterprise edition covers your specific use until the agreement says so. If the license does not fit, Apache-2.0 components such as RAGFlow or a custom retrieval layer are alternatives to evaluate.

Does DeployOpen issue SOC 2 reports for our product?

No. DeployOpen prepares systems, controls and evidence for compliance assessments, but independent auditors issue SOC 2 reports. DeployOpen can help you prepare the control descriptions, configuration evidence and access logging that an auditor will examine, and it can run security testing of the application and its tenant boundaries. Whether a SOC 2 Type 1 or Type 2 report fits your customers is a question for your auditor and your customers' procurement requirements. Ask a potential auditor which evidence period and system description they expect before the build starts.

How DeployOpen scopes and supports a customer-facing RAG assistant

DeployOpen is a Chainscore Labs company that plans, implements, hardens, documents and supports self-hosted open-source platforms inside the client's own infrastructure. For a customer-facing RAG assistant in a SaaS product, we start with a written scope covering tenant isolation, license fit, per-customer cost controls and answer evaluation, then build and hand over the system with the evidence your team needs.

How we scope

Engagements are fixed-scope, with scope, deliverables and timelines agreed in writing before work starts.
Scope defines which tenants, document sources and customer data types the assistant will index and answer from.
We confirm license fit for the chosen platform before build, including multi-tenant use limits that apply to the product.
The written scope sets the evaluation criteria, such as citation accuracy and refusal behavior, that must pass before release.

How we build

We deploy the chosen RAG platform and a self-hosted vector database in your cloud accounts or data centers.
Each customer's embeddings are isolated by tenant, with access checks enforced at retrieval rather than only in the interface.
We configure per-customer usage limits and logging so consumption can be attributed and capped for each account.
We build a repeatable evaluation set from representative customer questions and test retrieval and answers before each release.

How we hand over

Handover includes architecture documentation, tenant isolation design, configuration records and runbooks for your operations team.
We document the re-indexing process, including how deleted or updated customer documents are removed from the vector store.
Evaluation results, test records and license notes are delivered so your compliance and product teams can review them.
Your team receives walkthroughs of the admin consoles, key rotation steps and the logs needed for customer or auditor questions.

How we support

Ongoing support, upgrades and continuous testing are available as retainers after handover.
Retainers can cover platform version upgrades, security patching and rerunning the evaluation set after model or content changes.
Periodic security testing of the assistant, including prompt injection tests on customer-uploaded documents, can be scoped separately.
Support scope, response expectations and service levels are agreed in writing, and should be confirmed against your contract needs.