Skip to main content

Command Palette

Search for a command to run...

IP, Data, and What Your AI Vendors Are Really Doing With Your Inputs

Your vendor says they do not train on your data. What does that actually mean, what are they allowed to do, and what should be in your next contract? Here is the honest decomposition.

Updated
14 min read
IP, Data, and What Your AI Vendors Are Really Doing With Your Inputs

Bottom line. "We don't train on your data" is a load-bearing promise in every 2026 enterprise AI contract, and it is also the promise most easily misread. Vendors honour the literal version of the clause while doing things with your data that customers would be surprised by — abuse-detection logging, safety-classification caching, human review of flagged outputs, embedding storage, retention for debugging. None of these violate "no training" in the strict sense. All of them surprise customers when they find out. This briefing decomposes what your vendor is actually doing, names the four things to read carefully in any DPA, and gives you the three contract provisions worth negotiating into every new AI vendor agreement in 2026. 20 minutes of reading; one clarified position for your next vendor contract review.

L3.2 gave your GC the five questions. This briefing goes deep on the data-flow question specifically because it is the single area where leadership and legal are most often talking past each other. Everyone agrees "data shouldn't be used for training." Almost no two people agree on what that sentence actually covers.


What "we don't train on your data" usually means — and doesn't

Start with the specific language. A typical 2026 frontier-provider DPA contains something like:

Customer Data submitted via the API is not used to train or improve our foundation models, nor is it used to train or improve models for other customers.

Read literally, this clause does four things:

  1. Promises your data won't feed into the next version of the provider's base model.
  2. Promises your data won't leak into other customers' experiences through shared model weights.
  3. Says nothing about logging, caching, retention, human review, or short-term model operations.
  4. Says nothing about what happens to outputs your users generate (vs. inputs your users submit).

Each of the "says nothing" items is where the gap between legal-literal and customer-assumption lives. Vendors are technically compliant with the promise while doing things a customer might reasonably believe the promise precluded. The failure mode is not vendor bad faith — it is that customers and vendors have different models of what the clause covers, and nobody flags the difference until an incident surfaces it.

The things a modern AI vendor is doing with your data in 2026, even under a strict no-training promise:

  • Abuse detection logging. Inputs are logged briefly (typically 30 days) to detect abuse patterns — prompt injection attempts, policy violations, coordinated adversarial use. Vendors need to do this to run a safe API; the alternative is no abuse detection.
  • Safety classification. Inputs are run through safety classifiers that flag policy-violating content. The classifiers run automatically and don't constitute "training" but they do involve your data being processed by additional models.
  • Human review of flagged outputs. If a small fraction (typically <0.1%) of outputs trigger a safety flag, a human reviewer may see the input and output. Vendors disclose this in their acceptable-use policies but don't usually highlight it.
  • Embedding storage for caching and retrieval optimisation. Some features (prompt caching, similar-query deduplication) involve storing hashed or vector representations of your inputs for short periods.
  • Retention for debugging and incident response. If a call fails or produces unexpected output, vendors retain the relevant data for debugging — often 30-90 days.
  • Aggregate telemetry. Metadata about your usage (call volume, token counts, feature usage) flows to the vendor for billing and capacity planning. This is not your content but it is usage data.

None of these are "training." All of them involve your data flowing through vendor systems in ways the headline clause doesn't cover. The question is not whether your vendor does these things — they all do — but whether your contract and your customer communication accurately reflect them.

One headline "training" exclusion, six other ways your data is touched. Your customers may think the headline covers all of them. It doesn't. Your contract should.


The four things to read carefully in any AI DPA

Most AI vendor DPAs in 2026 are 15-40 pages. Most legal reviews of them take 2-4 hours. You, as a senior leader, don't need to review the whole thing — you need to spot-check four specific sections that contain 90% of the meaningful content.

Section 1: the scope of "customer data"

What to check: the DPA's definition of customer data. Is it "content submitted via API" only? Or does it explicitly include metadata, usage logs, feature telemetry, and derived artifacts (embeddings, cached intermediate results)?

Why it matters: a narrow definition lets the vendor classify everything outside the literal "content you submit" as not covered by customer-data protections. Metadata, which includes your usage patterns and potentially recognisable artifacts of your product, falls outside the narrow definition and gets weaker treatment.

What a good definition looks like: customer data includes all data provided by you or generated on your behalf, including inputs, outputs, intermediate results, embeddings, logs, and metadata directly attributable to your usage. "Directly attributable" is the key phrase that keeps normal aggregate capacity planning out of scope while pulling in your specific usage patterns.

Red flag: a definition that explicitly excludes "usage metadata," "system logs," or "derived artifacts." These carve-outs are where surprise-flavoured data handling lives.

Section 2: the retention policy

What to check: how long is data retained, in which locations, for what purposes, and how is deletion verified?

Why it matters: the difference between "30 days" and "30 days or until no longer needed for debugging, whichever is longer" is enormous. The first is a hard deadline; the second is "however long we want" in legal costume. The same applies to "until you request deletion" (reasonable) vs "indefinitely unless you take action" (not reasonable).

What a good retention policy looks like:

  • Maximum retention for inputs and outputs: 30 days for normal operation.
  • Shorter option available (zero retention, sometimes called "no retention" or "ephemeral") for sensitive workloads at higher tiers.
  • Exception for abuse investigations: up to 90 days, disclosed.
  • Deletion on customer request: confirmed within 30 days, with audit trail.
  • Backups cleared within standard backup rotation (typically 90 days).

Red flag: language like "we retain data for as long as necessary to provide the service" or "retention periods may vary by region." These sound reasonable but give the vendor unlimited retention with minimal disclosure.

Section 3: the sub-processor list

What to check: who does the vendor share your data with, for what purpose, and under what controls?

Why it matters: modern AI platforms are not monolithic. A single API call may touch the model provider's infrastructure, a cloud provider (AWS, GCP, Azure), a content delivery network, an observability vendor, a fraud-detection service, and a compliance-logging platform. Each is a subprocessor under GDPR and equivalent frameworks. Your customers' data is only as protected as the weakest subprocessor.

What a good subprocessor disclosure looks like: a complete list of named companies, their purpose (infrastructure, security, support), and their regional footprint. Updated when the list changes. Customer-notifiable changes if a material subprocessor is added or removed.

Red flag: a clause that says "we may update our subprocessor list from time to time" without notification requirements. Courts and regulators treat silent subprocessor additions as material.

Section 4: the training and improvement carve-outs

What to check: the specific language excluding training, and what exceptions or conditions apply.

Why it matters: this is the headline clause. Vendors vary in how strict their language is and how they describe the exceptions. A clause that says "we don't train our models on your data except as needed for operational purposes" has the exception wide enough to drive a model through. A clause that says "we don't train our foundation models or specialized models on Customer Data, full stop, and this applies to all sub-processors" is materially stronger.

What a good training exclusion looks like:

  • Explicit coverage of both foundation models and fine-tuned specialisations.
  • Coverage extended to subprocessors.
  • Explicit exclusion of operational uses that might be confused for training (abuse detection is fine; "model improvement" is not).
  • Audit rights: customer can verify compliance on reasonable notice.
  • Remedies if the clause is breached: typically termination + refund, sometimes penalties.

Red flag: "we don't train on your data except with your consent" or "opt-out to disable training." These formulations place the default on the wrong side and rely on the customer actively declining, which many don't until after the fact.


The three provisions worth negotiating into every new AI vendor contract in 2026

Most procurement conversations about AI vendors focus on pricing and SLA. Add three more to the mandatory list. Each is achievable — frontier vendors agree to these at the enterprise tier in 2026 — and each closes a specific gap that would otherwise become a problem later.

Provision 1: zero-retention option with documented verification

The ask: the vendor offers a "zero retention" or "ephemeral mode" for your API calls, in which no inputs or outputs are retained beyond the duration of the request. No logs, no caching, no debugging holdbacks. Signed attestation that the mode is enforced.

Why it matters: for sensitive workloads (healthcare, legal, financial data, highly regulated industries), even the default 30-day retention is too much. Zero retention eliminates the category of data exposure entirely. All major frontier vendors offer this mode at enterprise tiers in 2026; it is negotiable.

The gotcha: zero retention usually costs more (10-30% premium) and may exclude some features (caching, some rate-limit optimisations, some debugging support). Decide which of your features need it and scope accordingly. You don't need zero retention on everything — you need it on the high-sensitivity workloads.

Provision 2: customer-notified sub-processor changes

The ask: the vendor must notify you at least 30 days before adding a material sub-processor, with a right to object. Material means: any sub-processor that would process your customer data, or any sub-processor that would be added specifically for your account.

Why it matters: shadow sub-processor additions are how data-handling surprises happen. Your vendor adds a new analytics service to their stack; your data flows through it; nobody told you; six months later a security incident at the new service exposes customer data. You had no opportunity to object because the change wasn't notified.

The gotcha: the "right to object" is usually tempered — if you object, the vendor's remedy is to either forgo the new subprocessor for you specifically (rare) or allow contract termination (more common). Either is better than silent additions, but know what the exit is.

Provision 3: explicit language on human review of outputs

The ask: the vendor contract explicitly describes the circumstances under which human reviewers may see your inputs or outputs (e.g., for safety classification of flagged content), the volume involved, the data-handling protocols that apply, and the exception paths if you have higher sensitivity requirements.

Why it matters: this is the single largest "customer assumption gap" in 2026 AI contracts. Customers assume their data is never seen by humans at the vendor; vendors have reviewers who see flagged content for safety purposes. The gap only surfaces when a customer discovers it — usually through a routine audit or a media story — and is always awkward when it does.

What a good clause looks like:

  • Named circumstances under which review occurs (safety classification, abuse investigation, customer-requested debugging).
  • Approximate volume (e.g., "less than 0.1% of API calls may trigger review").
  • Controls on reviewers (trained, screened, under confidentiality).
  • Exception path: customers with higher sensitivity can contractually exclude their data from review, with documented impact on available features.

The gotcha: some safety classifications genuinely require human review in some fraction of cases for the vendor to run a compliant service. Excluding your data from all human review may preclude you from using the service at all in certain regulated jurisdictions. Negotiate the exception path carefully.


A worked example: the vendor contract audit that saved an enterprise deal

The setup: a 600-person fintech company is buying an AI-powered customer support tool. The tool uses a frontier model under the hood. Sales cycle is 4 months in. Contract is at legal review. Then the fintech's customer — a major US bank — runs their own diligence on the tool's data-handling story.

What the bank found: the tool's DPA had a narrow definition of "customer data" that excluded "system logs and operational metadata." Their interpretation: the tool vendor could, in theory, use the bank's data in ways not covered by the "no training" clause because it was technically "operational metadata" not "customer data."

The fintech's initial response: pass through the vendor's DPA unchanged and tell the bank it's fine. Bank's legal team refused to accept this. Deal stalled.

The fintech's real response (3 weeks later): the fintech's GC, reading this kind of guidance, did the four-section audit on the vendor's DPA. Identified the narrow definition as the specific issue. Negotiated a side letter with the vendor that:

  1. Broadened the definition of customer data to include metadata attributable to specific account usage.
  2. Added a 30-day customer notification requirement for new subprocessors.
  3. Offered the bank a zero-retention option for their account at no additional cost.
  4. Added explicit language about human review circumstances.

All four changes were negotiable because the vendor understood the alternative was losing a $2M+ deal. The side letter took 10 days to negotiate and draft. The deal closed. The bank's legal team signed off.

What would have happened without the audit: the fintech would have either lost the deal (reasonable concern) or signed the bank's extremely strict terms flat (reasonable concern). Either outcome was worse than the negotiated side letter.

The specific cost: ~15 hours of GC time on the audit + ~10 days of back-and-forth negotiation = $30K-$50K of legal and executive time. Against a $2M+ deal, the ROI is clear.

The generalisable lesson: AI vendor DPAs are negotiable at enterprise scale in 2026. Vendors agree to reasonable requests because they want the deal. The leverage is highest before contract signing and decreases after. Your legal and leadership team should treat DPA negotiation as part of the pre-contract work, not as a post-signing cleanup.


The specific pattern that wastes the most value at vendor contract time in 2026: treating the vendor's default DPA as a take-it-or-leave-it document and passing it through to your own customers unchanged. The pattern:

  • Your legal team receives the vendor's DPA.
  • They note that it looks standard.
  • They pass it through to your own customers as part of your terms of service.
  • Your customers' legal teams read it, find the gaps, come back with concerns.
  • You now have an awkward three-way conversation with the vendor who doesn't care about your customer's specific concerns and a customer who doesn't care that your vendor's DPA was "standard."

The "pass-through" feels efficient (no extra legal work) and is actually more expensive because the negotiation happens at the worst possible time (with your customer instead of with your vendor) and with the worst possible leverage (you're the middle party, with no direct relationship between the two parties who actually need to agree).

The defence: treat every AI vendor DPA as a document to be improved before you ever pass it through to a customer. The improvements from the "three provisions" section are usually agreeable to the vendor at enterprise tier. The improvements to "scope of customer data" and "subprocessor notification" are usually agreeable too. By the time your customer sees the final DPA, it should already be better than the vendor's default — because you improved it on their behalf, before they asked.

The cost is a few hours of GC time per vendor. The value is every enterprise deal that would otherwise stall on data-handling concerns closing faster.


What to decide on Monday

  • Audit your current AI vendor DPAs against the four sections in this briefing. Two hours per vendor. Flag gaps.
  • For each gap, list the specific concern in plain language. Not "the definition is narrow" — "the definition excludes metadata, which means the vendor could theoretically train on our usage patterns without breaching the no-training clause."
  • Negotiate the three 2026 provisions into every new AI vendor contract. Zero-retention option, sub-processor notification, explicit human-review language.
  • Do not pass-through vendor DPAs unchanged to your customers. Improve them first, then pass through the improved version.
  • Clarify with your platform team which workloads need zero-retention mode. Not all of them need it; the ones handling PII, health data, financial records, or regulated content usually do.
  • Maintain a vendor DPA diff log so you know which vendors you've already pushed to improve and which need another round. Part of the vendor inventory from L3.2.
  • Re-audit vendors annually. DPAs update; your inventory should too.

Next briefing, L3.4, closes Module L3 with the specific leadership-level question that L3.2 introduced but didn't resolve: what does AI incident response actually look like when the incident hits the news. The 48-hour playbook from the leader's perspective, the specific decisions you'll make in the first six hours, and the public communications that turn a crisis into credibility.


Course navigation

⬅️ Previous📍 You are hereNext ➡️
⬅️ Previous
L3.2 · Legal, Compliance, and AI Risk Surface
L3.3 of L4.3Next ➡️
L3.4 · Leadership AI Incident Response

📚 AI for Leaders · Course Home — 15 briefings, four modules.


Cover photo via Unsplash. This post is part of the AI for Leaders series.

More from this blog

Learn AI - Zero to Hero

111 posts