Docker Sandboxes Plug Harbor Into MicroVM Agent Eval Runs

Classy aggregates publicly available material; this is not a product review.

What changed

On 14 October 2026, Docker published an integration that lets the open Harbor evaluation framework run every task inside Docker Sandboxes: isolated micro virtual machines with their own kernel, optional network policies, and both local and cloud backends. Docker documents install via uv tool install "harbor[sbx]", cloud credentials through SBXDOCKERUSERNAME and SBXDOCKERPAT, and a nightly build pin harbor[sbx]==0.24.1.dev202610150414, with stable Harbor Sandboxes support targeted for 23 October 2026.

Harbor, maintained by the harbor-framework project behind Terminal Bench style benchmarks, orchestrates agent tasks, scoring, and logs. Sandboxes add the execution envelope so untrusted, model generated code cannot reach the host or other trials, and so evaluators can cut network access after dependency setup to reduce reward hacking.

Why it matters

Agent benchmarks only inform capital and safety decisions when scores are reproducible and hard to game. Running thousands of trials on a laptop or in CI without kernel level isolation invites cross trial contamination, credential leakage, and agents that pass checks by editing test harnesses rather than solving tasks. Docker’s post states that microVM isolation keeps each trial inside its sandbox and that configurable cloud network policy lets teams download packages during setup then restrict internet access while the agent works.

For applied AI leads, this is infrastructure for evidence grade evals: the same Harbor task definition can run locally with --ek target=local after sbx login, or scale on Docker Cloud Sandboxes with concurrent workers Docker says it tested up to 64 concurrent tasks.

Who is affected

ML platform and agent engineering teams building internal eval suites or shipping coding agents will feel this first. Security and red team groups reviewing agent tool use benefit from default isolation and explicit network modes. Benchmark authors in the Harbor ecosystem gain a documented sbx environment provider; upstream Harbor work tracked in pull requests such as #3366 and follow on #3536 on harbor-framework/harbor formalizes cloud and local backends, credential refresh, and Dockerfile caching.

Vendors of alternative sandboxes (Daytona, Modal, and others Harbor already supports) still matter, but Docker’s integration targets teams standardized on Docker accounts and Sandboxes billing.

What to do next

If you run Harbor or Terminal Bench style tasks today, pilot harbor[sbx] on a non production benchmark: one public task, one no network task, and one cloud run with PAT scoped to Cloud Sandboxes. Record failure modes (image build location, credential expiry, network policy gaps) before you tie hiring or release gates to scores.

What to watch

Watch for Harbor stable Sandboxes support around 23 October 2026, Docker Compose inside a single Cloud Sandbox for multi container evals, and whether benchmark maintainers publish recommended network policies per task suite.

Sources

• Primary — Docker, Harbor + Docker Sandboxes: Run Safe, Reproducible Agent Evals (14 October 2026). Announces microVM isolation, Harbor install commands, nightly package pin, and 23 October stable target.
• Primary — Docker, Docker Sandboxes documentation (accessed 17 October 2026). Defines local versus cloud sandboxes, sbx CLI, and credential separation.
• Secondary — harbor-framework, Harbor repository (accessed 17 October 2026). Framework scope, Terminal Bench harness role, and evaluation workflow.
• Secondary — harbor-framework, Pull request #3366: Docker Sandboxes environment (accessed 17 October 2026). Engineering detail on sbx provider, backends, and credential handling; superseded by ongoing integration work noted in the thread.