Non-deterministic auditing of AI agents is the process of recording the decision-making process and enabling post-hoc verification, based on the premise that agents may generate different outputs even for identical inputs.

Agents incorporating generative AI do not necessarily return the same answer to the same question every time. For this reason, an audit trail that can explain "why this particular decision was reached" after the fact becomes a prerequisite for quality assurance and incident response. This guide is intended for engineers and quality assurance personnel who are required to ensure agent accountability, and systematically covers the design and operational procedures for reproducible audit logs—from design premises through recorded items, anomaly detection, and failure patterns.

Traditional software could be debugged on the premise that "the same input produces the same output." However, in agents built around generative AI, this premise breaks down. We begin by examining the three structural factors that make auditing difficult.

In deterministic systems, reproducing a bug is achieved simply by "replaying the same input." Since identical inputs, code, and state yield identical outputs, the cause can be traced as long as the input is preserved in the logs.

Agents do not guarantee this reproducibility. Even when given the same prompt, a model selects tokens from a probability distribution, meaning the wording, conclusions, and order in which tools are invoked can vary. In other words, retaining only a "log of results" is insufficient—replaying the same input may produce a different result, causing a discrepancy between the log and the current behavior.

This requires a fundamental shift in how auditing is approached. What must be recorded is "what happened during that particular execution." The assumption that behavior can be verified by replaying inputs must be abandoned; instead, the starting point is to capture and fix all decision-making inputs at the moment of execution.

The primary source of non-determinism lies in the sampling used to select output tokens. Parameters such as temperature and top-p control whether only high-probability candidates are selected or whether lower-probability candidates are also given a chance. Raising the temperature increases output diversity, but also increases variance for identical inputs.

Lowering the temperature toward zero tends to stabilize outputs, but complete determinism is still not guaranteed. Minor differences can arise from backend parallel processing, the order of floating-point operations in hardware, and updates made by the model provider.

From an auditing perspective, it is essential to record the values of these parameters in the logs. Without logging temperature, top-p, seed values, and model version, the foundation for any post-hoc discussion of "why this particular output was produced" is lost. Rather than treating output variation as a fault, the practical approach is to ensure that all conditions giving rise to that variation are fully recorded.

For a single-turn response, output variation can be observed at a single point. The problem arises when an agent operates by chaining multiple steps together. In a structure where the output of one step becomes the input of the next, a small variation early on can significantly alter decisions in later stages.

For example, a slight change in tool selection at the first step can alter the data retrieved, potentially leading to an entirely different final conclusion. Since each intermediate step may appear to be a "reasonable" decision in isolation, examining only the final result makes it impossible to identify where the divergence occurred.

Auditing this chain requires recording all intermediate states between steps. By preserving the input, output, and selected tools for each step in chronological order, it becomes possible to trace backward from the final result and isolate which step was the point of divergence. A design that omits intermediate states and retains only the final result makes it impossible to trace this chain.

Before deciding on log fields, there are prerequisites that need to be established. Starting implementation without clarity on what to log, where to log it, and who is responsible for it tends to result in having to redesign the system later. Here we confirm three prerequisites to address before beginning the design.

The first decision is defining the boundary of what falls within the scope of auditing. Attempting to record every agent execution in equal detail makes costs and operational overhead impractical.

In practice, it helps to distinguish between high-risk operations and lower-risk ones. Steps involving external writes, payments, access to personal data, or irreversible actions should be logged in the greatest detail—from inputs through to the reasoning behind decisions. Conversely, processes that are primarily read-only and have limited impact may warrant only summary-level logging.

Defining scope is not a one-time decision. Whenever a new tool or permission is added to an agent, always review whether that operation falls within the audit scope. Expanding functionality while leaving the boundary design ambiguous risks allowing the operations most subject to accountability to slip through the logs unrecorded.

Audit logs derive their value from being accessible when needed, so the choice of storage destination and retention period design directly affects their quality. Mixing audit logs with application operational logs in the same location can result in older records being purged by rotation, leaving no trail when it matters most.

When selecting storage infrastructure, the criteria should be: the ability to write records in a tamper-resistant manner, ensuring searchability, and the long-term storage cost being justifiable. Writing to a write-only (append-only) area makes subsequent editing difficult to perform, increasing the reliability of the records as evidence.

Retention policies should be determined by working backward from business and regulatory requirements. Regulated operations may require retention spanning several years, in which case a practical design separates a fast-access hot tier from a low-cost archive tier to manage costs. Documenting upfront how long records will be kept and at what level of granularity simplifies operational decisions down the line.

Audit logging is both a technical mechanism and a matter of organizational accountability. It is necessary to decide before implementation who designs the logs, who monitors them on a day-to-day basis, and who is responsible for providing explanations when an incident occurs.

A common failure mode is that the development team believes "we are outputting logs," while the operations team believes "they were never shared with us as something to monitor," resulting in a state where no one is actually looking at them. Logs that are merely being output only function as evidence once a responsible party and procedures for referencing them are established.

When formalizing the division of roles, distinguish at minimum between three: the "owner of log design," the "person responsible for routine monitoring," and the "accountable party for explanations during an incident." Deciding in advance who will present and explain the reasoning behind an agent's decisions when questioned by external parties is what transforms audit logs from mere records into a means of accountability.

With the prerequisites in place, the next step is deciding specifically what to record. To support both reproduction and explanation, it is necessary to capture not just outcomes, but all of the inputs that informed the decisions leading to those outcomes, without gaps. Here, the items to be recorded are presented across three layers.

The first thing to preserve is the complete input passed to the model during execution. This means fixing exactly what the model actually "saw" — not just the user's query, but also the system prompt, referenced documents, past conversation history, and any variables embedded in templates.

One important point here is to avoid summarizing or reformatting the input before storing it. When reproducing or verifying behavior later, the raw text that was actually passed to the model carries a different meaning than a human-readable summary. When the basis for a decision is called into question, only the pre-formatted snapshot can be relied upon.

In architectures like RAG, where documents are retrieved from external sources and added to the context, the retrieval results themselves must also be logged. Even for the same query, the documents retrieved can vary depending on the timing of the search or the state of the index — so without recording "what was read at that moment," it becomes impossible to evaluate the validity of the output after the fact. When personal data is involved, design the logging in conjunction with encryption at rest and appropriate access controls.

Even with identical input, the output will vary depending on which model was queried and with what parameters. Therefore, for each execution, record the model name and version, sampling settings such as temperature and top-p, the seed value, and the maximum token count.

The model version in particular is easy to overlook. If the model provider releases an update, behavior can change even with the exact same prompt. When investigating a situation like "this was working correctly until last month," the investigation cannot move forward if it is unknown which version was in use at the time.

It is effective to record version information alongside the version of the agent's own code. Since prompt templates and tool definitions also change as part of the codebase, linking both the "model version" and the "agent implementation version" makes it possible to track which of the two changes caused a shift in behavior.

An agent's decisions depend heavily not only on the model's output, but also on the results of calls to external tools and APIs. Record sequentially, in the order they were made, which tool was called, with what arguments, and what was returned.

The key point here is to preserve the "arguments" and "return values" of each call as a pair. For example, if a tool was called to query inventory, both the queried ID and the returned stock count should be recorded. Without retaining the return values, it becomes impossible to reconstruct "what the agent saw when it made that decision," and the validity of the outcome cannot be verified.

External APIs require particular attention, as their responses change over time. It is normal for the same tool called with the same arguments to return a different result the following day. This is precisely why fixing the return value at that moment as an audit trail makes after-the-fact explanation possible. Exceptional behaviors such as failures, timeouts, and retries should also be retained at the same level of granularity.

Once the items to be recorded are determined, they must be structured in a way that allows them to be traced later. Simply outputting them as unstructured text makes it impossible to locate the relevant records when needed. This section covers three design principles for achieving both reproducibility and searchability.

A single user operation proceeds internally through multiple model calls and tool executions. To enable these to be reviewed together after the fact, issue a unique trace ID at the start of execution and attach the same ID to all related log entries.

With a trace ID, everything that happened in a given case can be gathered in chronological order simply by searching for that ID. Without one, fragmented logs must be pieced together using timestamps and usernames, and when multiple executions are running in parallel, mix-ups are likely to occur.

For multi-step agents, traceability is improved by maintaining a hierarchical structure with a trace ID (for the overall execution) and a span ID (for each individual step). Combining a single ID that spans the entire execution with IDs that identify each step makes it possible to navigate "where the branching occurred" as a tree structure.

Only by arranging logs in chronological order can the flow of decisions be understood. Each record should include a timestamp indicating when it occurred, captured at the highest possible precision. At second-level granularity, when multiple events occur at the same moment, it becomes impossible to determine their relative order.

However, timestamps alone may not be sufficient to guarantee causal ordering. When processing spans multiple servers, clock skew between those servers can cause events that actually occurred later to be recorded earlier.

To avoid this problem, in addition to timestamps, each record should carry a sequential number indicating the order of step execution, or an identifier referencing the preceding step. By recording "when it happened" and "in what order it happened" separately, causal relationships can be reconstructed without being affected by clock skew. An audit trail with disrupted ordering can lead to misidentification of branching points.

Outputting audit logs as human-readable text makes mechanical search and aggregation impossible as volume grows. Standardizing on a structured format—such as JSON Lines, which writes one event per line in JSON—makes subsequent analysis significantly easier.

The advantage of structured formatting is the ability to filter by individual fields. Operations such as extracting "only tool calls" "for a specific trace ID" "limited to failures" can be accomplished with a few lines of a query when the format is consistent. With free-form logs, the same task requires manual reading.

When defining the format, first fix the common fields—trace ID, timestamp, event type, target, and result—and group event-specific details beneath them. Standardizing the names and meanings of common fields from the outset allows logs to be handled uniformly across different tools and teams. Document the schema, and when changes are made, include a version number to maintain compatibility.

Simply retaining logs is not enough to notice when problems are occurring. A mechanism is needed to actively measure whether non-determinism has exceeded acceptable bounds and to detect anomalies. Here, we examine three key implementation considerations for quantifying variance and connecting it to alerts.

To understand the degree of non-determinism, the basic approach is to run the same input multiple times and measure how much the output varies. A single execution makes it impossible to distinguish whether that output is stable or merely an outlier that happened to appear.

In practice, select a set of representative inputs, run each repeatedly, and observe the distribution of results. If the output is nearly identical each time, the behavior is stable; if it varies widely, the agent's behavior for that input can be considered unreliable.

This measurement can also serve as a regression check when models, prompts, or tool definitions are changed. Run the same input set before and after a change and compare whether variance or conclusions have shifted in unexpected ways. While non-determinism cannot be eliminated entirely, knowing "which inputs are prone to variance" allows monitoring to be concentrated on high-risk areas.

When measuring output variance, relying solely on whether strings match can lead to a misreading of the actual situation. Different phrasing with the same conclusion is not a problem, whereas nearly identical wording with a single differing number or conclusion can represent a critical difference. What matters is capturing how much the outputs differ at the level of meaning.

One method for measuring semantic differences is to convert outputs into embedding vectors and assess similarity by the distance between vectors. This allows the closeness or divergence of content to be handled numerically, without being influenced by minor differences in wording. However, this approach is not foolproof—it can miss cases where outputs appear semantically similar yet reach opposite conclusions.

For this reason, differences that are critical to the business—such as final judgments, numerical values, or actions to be executed—should be extracted and compared strictly on their own, separately from semantic similarity. What constitutes "acceptable variance" and what constitutes "a difference that cannot be overlooked" must be defined based on the nature of the business.

Even if you build a detection mechanism, people cannot be watching logs around the clock. Design thresholds and escalation paths so that notifications are sent automatically when variance or anomalies exceed a certain level.

When setting thresholds, making them too strict causes notifications to fire continuously and get ignored, while making them too lenient means missing genuine anomalies. The practical approach is to start conservatively and adjust based on the notifications that arise during actual operation. The criteria for what counts as an anomaly can be defined not only as fixed values, but also as deviations from the distribution observed during normal conditions.

Escalation should be tiered according to the magnitude of impact. Separate the paths accordingly: minor fluctuations are logged only, notifications go to the responsible party when tolerance is exceeded, and for anomalies involving irreversible operations, processing is halted and human review is required. Anomaly detection only translates into prevention of real harm once you have designed not just the detection, but also who does what from the moment of detection onward.

Finally, two common implementation pitfalls are worth highlighting. Both tend to surface in the form of "it works, but is useless when it actually matters," so they are best avoided at the design stage.

The most common failure is logging only the final result while omitting the intermediate decisions that led to it. At first glance, knowing the result alone may seem sufficient. However, for agents with non-determinism, this design choice becomes fatal.

When you try to investigate a cause relying solely on result logs, the only option is to "replay the same input and reproduce the issue." But because an agent can return different results for the same input, replaying it does not reproduce the conditions at the time, and the investigation goes back to square one. If the intermediate steps — which tools were called, what was returned, and how decisions were made — are not preserved, no one can explain the validity of the result.

The principle for avoiding this failure is simple: abandon the assumption that "we can always replay it later," and instead fix and retain all decision inputs and intermediate states at the moment of execution. Cutting intermediate records to save on storage costs is a trade-off that means losing the audit trail precisely when accountability matters most.

Once you understand the importance of retaining intermediate steps, the opposite problem arises. Retaining everything at maximum detail causes log volume to grow rapidly, making storage costs and search performance impractical.

The fundamental way to handle this tension is to return to the concept of scope discussed earlier: apply fine granularity to high-risk operations and summary-level logging to low-impact processes. A design that retains everything uniformly at maximum granularity is unsustainable both in terms of cost and searchability.

In addition, tiering by retention period is effective. Keep recent records in a hot storage area that is easy to search, and move records past a certain age to low-cost archives. For large payloads — such as the full text of referenced documents — rather than embedding them directly in the log, another option is to store them separately and retain only a reference. Cost and sufficient auditability are not an either/or choice; treat them as a problem to be balanced through the design of granularity and retention periods.