PII Redaction for Agent Traces and LLM Logs: A Checklist

Where does sensitive data hide in an agent trace?

An agent copies whatever its tools return into the trace. Hugging Face's guidance puts it plainly: "Trace files can include prompts, tool inputs, command output, local paths, screenshots, secrets, private code, and personal data." (Hugging Face). The OpenTelemetry attribute registry warns that gen_ai.input.messages "is likely to contain sensitive information including user/PII data" and that gen_ai.tool.call.arguments "may contain sensitive information" (OTel registry).

What is the right order to redact an agent trace?

Work through the categories in this order on every run. The checklist (CSV) has the same steps with a column for who did each one.

• 1. Secrets: API keys, tokens, passwords, private keys and connection strings. Then revoke or rotate each one. GitHub's guidance: "It is not sufficient to simply remove the secret from your codebase. The most important remediation step is revoking the secret with the secret's provider." (GitHub Docs)
• 2. Personal data in prompts, messages, tool arguments, tool results and reasoning. NIST SP 800-122 defines PII to include "any information that can be used to distinguish or trace an individual's identity, such as name, social security number, date and place of birth, mother's maiden name, or biometric records" (NIST SP 800-122).
• 3. Customer and third-party content: records about your customers' own clients, and data from enrichment or market-data vendors. Remove it, or replace it with consistent placeholders so the workflow still reads.
• 4. Internal infrastructure: hostnames, IP addresses, internal URLs, file paths and usernames.
• 5. Confidential instructions: system prompts or tool descriptions that hold policy, pricing logic or anything under an NDA. Replace with a one-line description of what the instruction did.
• 6. Regulated data: health information to the HIPAA standard (see below), card data removed entirely, and privileged legal material excluded. Card verification codes are sensitive authentication data that "cannot be stored after authorization" (PCI SSC).

How do you pseudonymise traces and keep them readable?

Replace each person, account or customer with the same placeholder everywhere it appears in a run (Customer A, Engineer 2, ACCT-0007), so the agent's steps still make sense and joins between tool results still work. Keep the mapping table separate, access-controlled and out of anything shared.

Two rules from the standards. NIST: "The code, algorithm, or pseudonym should not be derived from other related information" (NIST SP 800-122), so no hashes of email addresses or initials. GDPR defines pseudonymisation as processing "in such a manner that the personal data can no longer be attributed to a specific data subject without the use of additional information, provided that such additional information is kept separately" (GDPR Article 4(5)), and data treated this way "should be considered to be information on an identifiable natural person" (GDPR Recital 26).

Pattern rules, NER or model review: which redaction method works?

None works alone. Each catches what the others miss, so run them in layers and sample the result.

Presidio's own documentation is candid: "because it is using automated detection mechanisms, there is no guarantee that Presidio will find all sensitive information. Consequently, additional systems and protections should be employed." (Presidio). The same holds for every tool in the table.

What goes in a redaction log?

A redaction log lets anyone check what was changed without seeing what was removed, and lets a run be re-processed when a rule improves. Record one row per run and entity type:

How many redacted traces should a person check?

No standard sets a sample size for trace review. A rule that works in practice, in Origin's view: a person reads every step of at least 20 runs or 5% of the set, whichever is larger, chosen at random, plus every run where any detector fired. Read tool results and reasoning, not only the prompts.

If the sample finds one missed secret or identifier, fix the rule, re-run the whole set and draw a new sample. Record the sample, the misses and the fix in the redaction log.

Which legal checks come before sharing redacted traces?

This page is general information, not legal advice. It quotes each clause and rule with its source and the date it was read; your counsel decides how they apply. A lawyer should read your own model-provider agreement (including any enterprise terms), your customer contracts and your privacy notices before any trace leaves the company.

Health data: HIPAA's Safe Harbor method requires that "The following identifiers of the individual or of relatives, employers, or household members of the individual, are removed: (A) Names; (B) All geographic subdivisions smaller than a State", through "(R) Any other unique identifying number, characteristic, or code", and that the covered entity "does not have actual knowledge that the information could be used alone or in combination with other information to identify an individual" (45 CFR 164.514). The same section sets out an expert determination method as the alternative.

• Customer contracts and data processing agreements: do they allow the content in tool results to be used or shared beyond providing the service?
• Third-party data terms: enrichment and market data inside tool results usually cannot be passed on.
• Employee notice: traces show who approved, who was paged and staff messages; works councils may need to be consulted in some EU countries.
• Privacy law: GDPR and UK GDPR treat pseudonymised data as personal data; US state privacy laws cover their residents' data.
• Model-provider terms: if the traces will be used to train or evaluate a model, read the provider's restrictions on using outputs.

Worked example: one tool result before and after redaction

Fictional data. An accounts-payable agent looked up a vendor and the tool returned:

{"tool": "erp.get_vendor", "result": {"vendor": "Northfield Supply LLC", "contact": "Dana Reyes", "email": "dana@northfield.example", "iban": "GB00 TEST 0000 0000 0000 00", "notes": "Pay before 15th, approved by J. Moss", "api_host": "erp-prod-02.internal.example", "token": "EXAMPLE-NOT-A-REAL-TOKEN"}}

{"tool": "erp.get_vendor", "result": {"vendor": "Vendor A", "contact": "Person 1", "email": "person1@vendor-a.invalid", "iban": "[BANK_ACCOUNT_REMOVED]", "notes": "Pay before 15th, approved by Approver 1", "api_host": "[INTERNAL_HOST]", "token": "[SECRET_REMOVED]"}}

Redaction log rows for this step: one secret (token, pattern rule, removed, key revoked), one bank account (pattern rule, removed), two person names (NER, replaced), one email (pattern rule, replaced), one hostname (pattern rule, replaced). The run still shows the agent finding the vendor, reading the payment note and moving on, which is what makes it usable.

Common mistakes when redacting LLM logs

Each of these has let sensitive data through or made a run unusable:

• Scanning prompts and final answers but not tool results or reasoning.
• Deleting a leaked key from the log without revoking it.
• Replacing the same person with different placeholders in one run, so the steps no longer make sense.
• Deriving pseudonyms from the real value (hashing an email address).
• Redacting so much of each tool result that nobody can tell what the agent read.
• Skipping screenshots and attachments.
• Keeping no log, so nobody can show what was removed or re-run an improved rule.

Download the checklist and the data card template

Agent trace redaction checklist (CSV): each step in order, where to look, the method, how to verify it and columns for owner, date and notes. Agent trace data card template (CSV): one row per field, including the redaction log and reasoning visibility, with two fictional example runs. Both open in Excel or Google Sheets.

To pull the traces out of your tracing tool first, see how to export agent traces.

Redacted traces are also a record of how work gets done

A redacted, logged set of complete runs is useful well beyond one review: for evaluation, for fine-tuning your own models and, where the contracts allow, for licensing.

Thinking of licensing your traces? The guide to licensing agent traces covers what AI developers ask for, the provider terms and who can sign, and you can describe your agent traces without sending files.