Customer Data and AI: What You May Legally Put Into a Model, and How to Prove It

Most companies that want to use AI on customer data ask one question: is this allowed? It is the wrong question. The one that decides things is whether you can show, a year from now and to a regulator or a large client, that it was allowed and that you handled it properly. The legal work is mostly paperwork and discipline. The good news is that you can plan for both.

This article uses the GDPR as its reference point, because it sets the bar most other privacy laws copy. It is not legal advice. It covers the decisions your lawyers will ask you about, and it is better to have answers ready before they ask.

Start with purpose

Customer data is collected for a reason: to fulfil an order, provide support, send an invoice. The law generally lets you use it for that reason and for purposes compatible with it. Using support tickets to help your support team answer faster is usually compatible. Using the same tickets to train a model you plan to sell to other companies usually is not.

So before anyone uploads anything, write one sentence: we are using this data to do X, for the benefit of Y. If you cannot write it, or it reads badly, stop there. That sentence becomes the anchor for everything else.

Know your legal basis

Every use of personal data needs a lawful basis. For AI projects the realistic options are usually contract (the processing is needed to serve the customer), legitimate interest (you have a genuine business need that does not override the customer's rights), or consent.

Legitimate interest is the most common basis and the one most often claimed without the work behind it. It requires a written balancing test: what you gain, what the customer risks, and why they would reasonably expect this use. That document takes an afternoon to write, and it is the first thing an auditor will ask to see.

Some data does not go in

Health information, biometric data, religious or political beliefs, sexual orientation, trade union membership: these are special categories, and the bar for processing them is much higher. The same caution applies to children's data and to financial identifiers such as card numbers.

The practical rule is simple. Unless the whole project exists to handle that kind of data, and you have taken advice on it, filter it out before it reaches any model. Redaction tools exist for this. Test them against your real data, because a filter that catches 95% of card numbers still lets the other 5% through.

Where the data goes matters as much as what it is

Sending customer data to an AI vendor makes that vendor your processor. You need a data processing agreement with them, and you need to read three clauses closely:

• Training: does the vendor use your inputs to improve its own models? Business tiers usually say no. Get it in writing.
• Retention: how long do they keep your prompts and outputs, and can you shorten that?
• Location: where is the data processed? Transfers outside the EU need a legal mechanism, such as an adequacy decision or standard contractual clauses.

Running models on your own infrastructure removes most of these questions. It does not remove the need to document what you did.

Retrieval is easier to defend than training

There are two broad ways to give a model your data. You can train or fine-tune it, so the data becomes part of the model. Or you can keep the data in your own systems and let the model look up what it needs at the moment it answers, which is usually called retrieval.

For customer data, retrieval is almost always the safer choice. When a customer asks you to delete their data, you delete the record and it is gone from every future answer. Getting data back out of a trained model is, in practice, close to impossible. That makes the right to erasure very hard to honour, and regulators know it.

How to prove you did it right

Compliance you cannot evidence counts for little. Keep these records, and keep them current:

1. A record of processing listing the AI system, the data it uses, the purpose and the legal basis.
2. A data protection impact assessment. New technology combined with large volumes of personal data almost always triggers one. It is the single most persuasive document you can hand an auditor.
3. Signed processor agreements with every vendor that touches the data.
4. Logs showing what data entered the system, when, and under whose authority, plus evidence that your redaction ran.
5. A named owner who reviews all of the above whenever the system changes.

The EU AI Act adds its own documentation duties, especially where a system is used in high-risk areas such as hiring or credit. If your records already cover these five points, you are most of the way there.

Carry the weight early

Companies get into trouble when the pilot ships first and the paperwork is reconstructed later. Do it the other way round. Write the purpose, choose the basis, filter the data, sign the contracts, turn on the logs, and then build. It adds a few weeks at most, and it means the answer to "can you prove it?" is a folder, not a scramble.

That groundwork is a large part of what we do at Hercules A. Whether you work with us or not, do it before the first record goes in.