Tamper‑Proof Audit Trails for LLM Applications

The Problem

When AI applications handle sensitive data in healthcare, finance, or government, regulations like HIPAA, SOC 2, and the EU AI Act require organisations to prove exactly what happened with each piece of data. Traditional application logs capture API calls, but they don’t record the actual conversations between your application and AI models like OpenAI or Anthropic.

Without tamper-proof records of these AI interactions, organisations face compliance gaps and potential penalties. That’s where Traceprompt comes in.

What Traceprompt Does

Traceprompt is an open-source SDK that automatically creates tamper-proof audit trails for every AI conversation. With just two lines of code, it wraps your existing OpenAI, Anthropic, or other LLM calls to create encrypted, verifiable logs that auditors can trust.

How It Works

Understanding how Traceprompt works requires looking at the complete journey of an AI interaction - from the moment you call an LLM to when that interaction becomes part of an immutable audit trail. Let’s walk through this process step by step.

Client-Side Encryption: Your Data Stays Yours

The foundation of Traceprompt’s security is that your sensitive data never leaves your infrastructure in plaintext. When you wrap an LLM call, the SDK captures both the prompt you send and the response you receive. Before transmitting anything to Traceprompt’s servers, it encrypts this data using AES-256-GCM encryption with a key that only you control.

This encryption happens using your own AWS KMS key - what is referred to as call “Bring Your Own Key” (BYOK). The SDK generates a fresh encryption key for each interaction, encrypts your data with it, then encrypts that key using your AWS KMS key. This means even if someone gained access to Traceprompt’s systems, they would only see encrypted data that they cannot decrypt without your permission.

Creating Cryptographic Fingerprints

Once your data is encrypted, Traceprompt creates a unique cryptographic “fingerprint” of the entire interaction using the BLAKE3 hashing algorithm. This fingerprint, called a leaf hash, represents exactly what happened in that specific AI interaction. Any change to the original data - even changing a single character - would produce a completely different fingerprint.

These fingerprints are then linked together in a hash chain, where each new entry includes a reference to the previous entry’s fingerprint. This creates an unbreakable sequence - if someone tries to alter or delete an entry, the chain breaks and the tampering becomes immediately obvious.

The Anchor System: Publishing Proof to the World

Every few minutes, Traceprompt’s anchor service collects all the recent interactions and organizes them into a mathematical structure called a Merkle tree. Think of this like creating a table of contents for a book - the Merkle tree provides a single “root” hash that represents all the interactions in that batch, along with cryptographic proofs that any specific interaction was included.

Here’s where it gets really powerful: these Merkle roots are published to a public GitHub repository with GPG-signed commits. This creates a public, timestamped record that anyone can verify. If you ever need to prove that a specific AI interaction happened at a particular time and wasn’t tampered with, you can use the Merkle proof and the public GitHub record to demonstrate this mathematically.

Smart Privacy Protection

While your actual conversations remain encrypted, Traceprompt extracts and analyzes metadata that’s safe to store in plaintext. The SDK automatically scans for sensitive information like social security numbers, credit card numbers, and personal health information, categorizing the risk level as general, sensitive, or critical.

This metadata includes information like token counts, response times, which AI model was used, and what types of sensitive data were detected - but never the actual content. This allows compliance teams to generate reports and filter interactions (like “show me all conversations that touched protected health information”) without ever exposing the sensitive data itself.

Integration That Just Works

The remarkable thing about this entire process is how invisible it is to your application. Adding Traceprompt requires just a few lines of code:

import { init, wrap } from "@traceprompt-node";
import OpenAI from "openai";
import { config } from "dotenv";

// Load environment variables
config();

// Initialize Traceprompt once
await init();
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Wrap your existing LLM calls
const trackedChat = wrap(
  (prompt) => openai.chat.completions.create({
    messages: [{ role: "user", content: prompt }],
    model: "gpt-4o",
  }),
  {
    modelVendor: "openai",
    modelName: "gpt-4o",
    userId: "alice",
  }
);

// Use exactly as before - your app doesn’t change!
const response = await trackedChat("Hello, world!");

Your wrapped function returns exactly the same data as the original function would, so your application logic doesn’t need to change. The encryption, hashing, and transmission all happen asynchronously in the background with minimal performance impact - typically under 2 milliseconds of overhead.

The Complete Traceprompt Workflow

Getting started with Traceprompt and achieving full audit compliance follows a simple 4-step process that takes you from SDK installation to verifiable audit trails:

Dashboard Overview

Once your SDK is installed, you’ll have access to a comprehensive dashboard that provides real-time visibility into your AI interactions. The dashboard automatically categorizes PII detected in your conversations, tracks audit activity, and provides tools for generating compliance reports.

The Traceprompt dashboard provides real-time monitoring of AI interactions, PII detection, and audit trail generation

From Logs to Audit Packs

When it’s time for an audit, Traceprompt automatically generates comprehensive audit packs - cryptographically sealed ZIP files containing everything an auditor needs to verify your data’s integrity. Each audit pack includes:

• Encrypted interaction data - Your actual AI conversations, encrypted with your own AWS KMS keys
• Merkle proofs - Mathematical evidence that each interaction is part of the anchored batch
• GitHub anchor references - Links to public commits containing the Merkle root hashes
• Digital signatures - GPG-signed manifests proving the pack’s authenticity
• Verification tools - Scripts and metadata to independently verify all cryptographic claims

The Anchor System: Your Public Proof

Behind the scenes, Traceprompt’s anchor service continuously processes your encrypted logs through a sophisticated batching system. Every few minutes, it groups recent interactions into Merkle trees and commits the root hashes to a public GitHub repository with GPG-signed commits.

This creates an immutable, publicly verifiable timestamp for when your data was recorded. The anchor system processes thousands of interactions efficiently while maintaining cryptographic integrity through hash chains and Merkle proofs.

Independent Verification

The verification center provides auditors with tools to mathematically prove data integrity without requiring trust in Traceprompt. Verifiers can check digital signatures, validate Merkle proofs against public GitHub commits, and decrypt data using your own keys - creating a level of trust that traditional logging systems simply cannot provide.

Real-World Use Cases

Why This Matters

As AI becomes more integrated into critical business processes, regulators are requiring organisations to prove their AI systems are operating correctly and handling data responsibly. Traditional logging isn’t sufficient because:

• Regular logs can be easily modified or deleted
• They often don’t capture the actual AI conversations
• Sensitive data isn’t properly protected during storage
• There’s no way to verify the logs haven’t been tampered with

Traceprompt solves all of these problems by creating immutable, encrypted, and verifiable records of every AI interaction - turning compliance from a burden into a manageable, automated process.