How Should Data Security Be Handled in LLM Integrations?

A support agent clicks the button that summarises a customer's long message. Asked where that message goes, most teams answer in one word: the provider.

True, but incomplete. The message doesn't go somewhere — it multiplies.

Security discussions usually skip past that multiplication and land on "which provider is safer." In practice the deciding factor is how many copies of the same data remain in your own system, and who decided on each one.

How many copies does one sentence have?

Following a single call step by step teaches more than an abstract security conversation. Summarising that customer message typically produces these copies:

The prompt body. The message travels to the provider wrapped in your instructions. This is the visible copy, and the one everybody knows about.

The provider's request record. How long the request is retained on their side depends on your account type and configuration. The default is not the same everywhere.

Your own application logs. This is the most commonly missed copy. Prompt and response are logged to make debugging easier, the line becomes permanent, and months later nobody remembers there is customer data in it.

The error tracker. When an exception occurs during the call, most tools collect the request body automatically. Nobody decided to store customer messages in an error tracker; it simply happens the first time something throws.

The cache. A cache added to avoid reprocessing the same input stores, by definition, the input itself.

The evaluation set. Real examples are set aside to measure quality. This is often the longest-lived copy, and it usually sits in an engineer's directory.

Review screens and screenshots. Every output a human inspects appears on a screen, and circulates through support threads as an image.

The length of the list isn't the surprise. The surprise is that most of these copies arrive as side effects rather than product decisions. So the first step in securing an integration isn't adding a control — it's taking inventory: where does this data currently sit?

The cheapest security decision is not sending it

Once the inventory exists, the most effective move is usually not a technical one. A field you never send is never stored, never logged, never leaked, and never needs deleting.

That means reviewing the prompt field by field: which of these does the model actually need to do the job? To summarise a message, the customer's name, email, phone number and account ID are usually irrelevant — the summary doesn't rest on them. The whole record gets sent because sending the whole record is easier.

The second move is making the necessary fields unidentifiable. If the model needs an identifier, you can send a placeholder that means something only inside your system and substitute the real value back when the response arrives. It doesn't fit every scenario, but where it fits it ends the argument.

This is a scoping decision more than a security measure, and it continues from whether the product needs AI at all: when you decide what context the model requires, the cost of that context includes the burden of storing and protecting it.

Retention isn't decided in one place

"We keep data for ninety days" sounds precise. Ask which copy is kept for ninety days and the answer scatters.

The log infrastructure's retention was probably set by an infrastructure engineer years ago for an entirely different reason. The error tracker's comes with the plan you bought. The cache's lifetime was chosen for performance. The evaluation set often has no lifetime at all, because nobody decided to delete it.

So there is no single retention policy — there are five, none of them aware of the others. This becomes visible the moment a customer asks for their records to be removed: deleting from the main database is easy, and the rest is only possible if it was planned for.

The practical approach is to write retention down per copy. It doesn't need a long document; three lines each will do: how long it stays, who deletes it, what happens if nobody does.

Access gets overlooked more than retention

As important as how long data sits is who can see it while it does.

The common picture: customer data is protected inside the product by role-based permissions, while the same data sits in plain text in a log viewer open to the whole engineering team. Careful authorisation on the product side is undone by a back door.

The question here isn't about tooling. Can you state who has access to each copy? If you can't, access control effectively doesn't exist.

The provider relationship is a configuration question

Providers are usually compared on model quality, but from a data standpoint what matters is how the account is configured. The questions to settle before reading any contract: is your data used for training, and which way does that behave by default; how long are requests retained on their side; in which region is the processing done; does the provider rely on other services underneath.

None of this substitutes for legal obligations — what regulation requires is a separate discipline and belongs with your legal counsel. The aim here is more modest: knowing that an integration shouldn't start before these items have been discussed.

The most frequent mistake in practice is administrative rather than technical: an API key created on a developer's personal account during the prototype, carried into production because it was already there. From that point on, company data flows through an account where none of the organisational settings apply.

Leaks don't only happen on the way in

The discussion is usually built around what you send the model. What it returns is just as much a data path.

In systems that assemble context — arrangements that gather pieces from your own records to answer a question — the real risk is putting the authorisation filter in the wrong layer. Asking a model to "only use records this user is allowed to see" is not a security control. Authorisation belongs in the query that fetches the data; a model cannot leak a record it was never given.

The second point is features where the user shapes the prompt's contents. In anything that accepts free text, incoming text may behave like an instruction. There is no complete fix, but limiting what the output is permitted to do — not letting the model take actions directly — reduces the risk concretely. That comes from the same place as deciding where human oversight sits.

What are you telling the user?

The last and most deferred question: does the user know their message goes to a model?

Saying so costs less than most teams assume. A single clear sentence lands far better than the same fact being discovered later — and the logic from how trust is built applies here too: institutional trust comes from verifiable behaviour, not from claims.

In closing

Data security in LLM integrations is too distributed a subject to reduce to choosing a provider. The risk usually comes not from the provider but from the copies the data leaves along the way — the ones nobody decided on.

So the most productive start is not adding security tooling but taking inventory: how many places does this data sit, how long, who sees it. Once the count exists, most of the measures suggest themselves — and a large share of them amount to never sending the unnecessary field.

It is worth considering this alongside the next step, moving a prototype into production, because most of those copies appear during exactly that transition. In our AI integration work, mapping the data flow comes before choosing a model.