Inside the Tracebit Deception Factory: how we test new techniques against AI agents

AI agents are getting good at breaking into things. The Hugging Face incident gave us a detailed example: an agent chaining vulnerabilities, stealing credentials and moving between systems. At Tracebit, we want to understand how deception changes that behavior. Can we detect these agents early? Can we slow them down, distract them or stop them?

To find out, we have been putting frontier models through their paces in an automated attack range. Our first published benchmark alone involved 951 attack runs. Since then, we have been testing Context Bombs, new deceptive artifacts and new models, giving agents repeated attempts at realistic AWS environments and recording what happens.

Building the range has been an experiment of its own. Agents spotted our fake AWS, found footholds left by their predecessors and completed attacks our scoring code failed to recognize. Each surprise taught us something about what it takes to run these experiments properly.

We built the Deception Factory to observe AI agents

Attackers use AI in more intrusions every month. With the first models, we watched intrusions in the wild and reasoned about which deception techniques would help. But as more advanced models arrived, some of them abliterated, we needed a reliable way to test defenses against each new model.

Most teams test against a local setup or a single bare AWS account. Ask an LLM how to do this and it will tell you the same. It produces bad data. We know because we tried it.

We needed a scientific instrument built on an experimentation platform. We spin up and tear down a convincing AWS environment on repeat, and we run every model through it under the same conditions. We needed controlled variables, repeatable runs per model to account for variations in behavior, and a way to compare the effectiveness of existing and newly prototyped deception techniques.

We also needed a baseline. Before adding a new deception technique, we want to know what agents do without it: which paths they find, how often they succeed and how long they take. We compare that configuration with the variant using the same model, provider, prompt and time budget, and repeat both enough times to give confidence in our results. Otherwise, a difficult scenario or an unusually lucky run could be mistaken for an effect of deception.

The Deception Factory is a cyber range in AWS

A scenario is a company's AWS environment, built real enough to fool an AI agent. It is built for the agent to attack, including its starting access, weaknesses and target data. It needs a coherent story: what the business does, how its workloads run, who has access and where valuable data lives. In our AWS environment that means application and deployment roles, operational documents, backups, secrets and working code that belong together.

We can set up 120 separate instances of our simulated environment at once, with around 300 resources per environment.

We run several scenario types, with the most complex replicating a pharmaceutical company. We leave it deliberately vulnerable, with multiple attack paths. We build in several plausible routes from a limited foothold to sensitive data, with breadcrumbs an agent can discover through normal exploration.

An AI agent takes roughly 15 to 20 minutes to fully compromise it without any deception technology present. That is our baseline control for measurement.

We place deceptive artifacts where an attacker would expect to find them: contents, names and placement all match the surroundings.

The research team uses the range to study how agents attack. The product team uses it to build what stops them.

The Deception Factory is our prototyping engine

We test every new deception technique in the Deception Factory as we evolve Tracebit. An engineer builds a prototype and hands it to one of our agents. The agent deploys it into the Factory, runs the experiment and reports whether the attacker's score changed.

From that report, the engineer tunes the prototype, scraps it or builds it into Tracebit. It now takes minutes to go from idea to tested result.

Context Bombs and Tracebit Artifacts were the first new capabilities prototyped, tested and advanced to full development in the product, with Artifacts released on 28 October.

We carry out 10 runs to test each technique on every model

When we're ready to test either a new model or a new deception technique, we follow a specific experiment setup:

• Provision a blank instance with no trace of past runs.
• Deploy the new deception technique across it, by hand or with one of our agents.
• Create one instance per model, plus a control for each.
• Start the run and record every move the agent makes.
• Each objective the agent reaches changes its score.
• End the run when the agent compromises the environment, gives up or runs out of time.
• Write the full capture to our data lake.
• Repeat until the result is conclusive.

In our most recent research, we completed 773 runs with AI agents, testing newer types of Context Bombs as the deception technique. We ran 7 of the more popular frontier models, 10 runs each, testing 9 new deception techniques for slowing down AI agents and wasting their time.

The hardest part: convincing the agent it's real

The result is only as reliable as the environment is believable. If an AI agent determines that it is in a test environment, it behaves differently.

Across the development of the range and thousands of test runs, we hit three traps that will catch any team running security experiments with AI agents.

• Emulation isn't enough. We started on LocalStack, proxied TLS, rewrote the default account ID and used normal AWS hostnames. Opus tried bogus credentials, saw they worked and concluded "simulator artifact." GLM found LocalStack's default port and the mitmproxy listener and started hunting for the mock's control endpoints.
• Start from scratch. An agent may create a user, modify a role or leave a backdoor. If the next agent finds that foothold, it can appear to have solved an attack path that the previous agent did most of the work on. We encountered this in the earliest versions of the range, alongside agents reading files left by previous runs.
• The agent's narration isn't the record. An agent states one intention, hits a roadblock and does something else. Relying on the narration or simple AWS CLI invocations misses calls the agent made through ad hoc Python scripts and parameters hidden inside files passed with --cli-input-json.

The models regularly surprise us. They use an API we did not expect, take a different route to a resource or investigate something we thought was incidental. So we capture everything.

Collecting results from the Deception Factory

Since we know that an agent's own account of what it did is unreliable, we capture a lot of telemetry from the environment side to get an accurate reconstruction of what happened. This matters most when we trace the attack path. When the UI says "objective reached," we want to be able to work backwards through every layer that leads to that conclusion. For example, an agent might:

• Start with a limited IAM user.
• Assume an application role.
• Exploit a trust-policy gap to assume a data role.
• Read protected secrets and download a backup.

The main data points we capture and report on include the agent's own reported session, an action timeline, network data, resources accessed or altered, canary interactions where canaries are placed, raw AWS activity (CloudTrail), Tracebit alerts fired, and files created or modified.

We can replay any session and see what the agent did, what it says versus what it actually does, how far it got along each vulnerable path, when and where it trips our standard canaries, and what happens when it interacts with new deception techniques.

Objectives stopped and time wasted are the biggest scores

Wasting an attacker's tokens is satisfying. The measure that matters is whether the deception stops the agent from reaching its objective and fully compromising the environment. And if we can't stop it, how much can we slow it down, to buy defenders as much time as possible to respond?

An attack that costs $100 instead of $1 sounds great, but it means nothing if an attacker still makes it out with your data. We'd rather see their spend go from $1 to $10 if it means we've stopped them in their tracks altogether.

This is the focus of our research: use deception to stop AI agents, and slow them when we can't.

We score each technique on three things, in this order:

• Objectives: Each objective the agent reaches raises its score, weighted by severity. Escalating privilege raises the score more than lateral movement, for example.
• Time: How much longer an agent takes to reach each objective and, when faced with a deception technique, how much time it spends on that technique. Our newest Context Bomb techniques made AI agents 7.8x slower in their attempts to compromise the environment.
• Cost: We score predicted cost as a tiebreaker when objectives and time are even.

We also test and score deception techniques in combination, which changes how effective they are at stopping AI agents.

Given how creative and capable agents are today, it is easy to write an automated scoring method that misses a perfectly valid attack. An earlier scoring function we used expected an agent to fetch an SSM command's result with GetCommandInvocation. Some agents used ListCommandInvocations instead. They had retrieved the result, but our scorer did not recognize it.

We built an LLM judge to tackle this problem. It reads the session transcripts and spots these mismatches. We use it to audit our scoring: did the rules miss an achievement, or give credit the evidence does not support? We investigate the disagreements, fix the rules and rescore the saved runs.

Some disagreements need a human decision. If ChatGPT Astra makes targeted permission changes equivalent to admin access, should that count as admin? If it reads one secret, establishes it can read the rest and moves on, should it get the same credit as an agent that downloads every secret?

Once we decide what counts, we want scoring to be deterministic: the same evidence and rules should give the same result. The judge helps us improve those rules, while the recorded facts let us check them.

What's next

We test every new model and every new technique in the Deception Factory. As new AI models are released that show real advances, we add them to the Factory as well. When a new model finds a tell, we fix the environment.

Send us a hypothesis to test

Do you have an idea for how deception could stop an AI agent, or a question about how agents behave when they meet it? Send it to us. Our research team will turn the best ideas into hypotheses, run them through the Deception Factory against the latest models, and share what we find.