Skip to main content
This page tracks all local benchmark suites shipped in benchmarks/:
  • bench (cold container spawn benchmark)
  • bench:pool (warm-pool benchmark)
  • bench:detailed (phase-level breakdown)
  • bench:tti (ComputeSDK-style TTI, cold)
  • bench:tti:pool (ComputeSDK-style TTI, warm pool)

​
Latest run snapshot

Run date: 2026-02-19 Environment capture time (UTC): 2026-02-19T09:28:47Z

​
Benchmark environment

ComponentValue
Host OSmacOS 26.2 (build 25C56)
KernelDarwin 25.2.0 (arm64)
CPUApple M3 (8 cores)
Memory16 GB
Container runtimeDocker Server 28.5.2 via OrbStack (linux/aarch64 guest)
Bun1.2.20
Node.jsv23.0.0
These benchmarks are host-sensitive. Re-run on your target machine before using values as SLO or regression gates.

​
bun run bench (cold spawn benchmark)

RuntimeMinMedianMaxAvg
python206ms220ms476ms301ms
node205ms211ms235ms217ms
bun190ms194ms214ms199ms
deno207ms223ms239ms223ms
bash181ms188ms189ms186ms

​
bun run bench:pool (warm pool benchmark)

RuntimeColdWarm avgWarm minSpeedup
python323ms132ms126ms2.6x
node237ms142ms132ms1.8x
bun213ms126ms120ms1.8x
deno232ms150ms144ms1.6x
bash192ms117ms113ms1.7x

​
bun run bench:detailed (phase breakdown, cold path)

RuntimecreatestartwritemkExecruncleanuptotal
python75ms57ms19ms1ms20ms45ms217ms
node45ms36ms14ms1ms28ms31ms154ms
bun46ms33ms12ms1ms16ms26ms134ms
bash38ms39ms13ms0ms12ms34ms136ms

​
ComputeSDK-style TTI benchmark

This suite is aligned to the ComputeSDK benchmark README, where TTI is measured as create/init + first command execution.

​
bun run bench:tti (cold, median TTI)

RuntimeMedianMinMax
bash0.18s0.18s0.19s
bun0.19s0.19s0.20s
python0.20s0.20s0.25s
deno0.21s0.21s0.21s
node0.21s0.20s0.22s

​
bun run bench:tti:pool (warm pool, median TTI)

RuntimeMedianWarm minWarm avg
bash0.12s0.11s0.12s
bun0.12s0.12s0.12s
python0.13s0.12s0.13s
node0.13s0.13s0.13s
deno0.15s0.14s0.15s

​
Comparison context vs ComputeSDK benchmarks

ComputeSDK’s README reports direct-mode provider medians (last run shown there: 2026-02-19T00:30:31.834Z):
SourceMedian TTI range
isol8 local (this page, warm pool)0.12s-0.15s
ComputeSDK direct mode providers0.29s-2.80s
This is directional context, not a strict apples-to-apples ranking. Environment, provider architecture, geography, and sandbox lifecycle semantics differ.

​
Reproduce locally (all suites)

# Cold spawn benchmark
bun run bench

# Warm-pool benchmark
bun run bench:pool

# Detailed phase breakdown
bun run bench:detailed

# ComputeSDK-style TTI (cold)
bun run bench:tti

# ComputeSDK-style TTI (warm pool)
bun run bench:tti:pool

​
Run benchmarks on GitHub Actions

You can also run benchmarks on a GitHub-hosted runner by manually triggering production-test.yml.
  • workflow: Production Tests (Manual)
  • runner: ubuntu-latest
  • input: set runBenchmarks=true
  • behavior: benchmarks run after production tests pass (the workflow executes bun run bench:cli)

​
FAQ

They represent steady-state latency after initial startup, which is the dominant user experience in long-lived services.
Use bench:detailed to find phase bottlenecks, then validate impact with bench:pool and bench:tti:pool.
Cold runs include one-time lifecycle costs that warm pools avoid (container creation/start and first-path setup).
Treat it as directional only. Use the same region, host class, and iteration settings before drawing hard conclusions.

​
Troubleshooting quick checks

  • benchmark command fails on first run: rerun sequentially, not in parallel
  • unexpectedly high warm TTI: verify pool defaults in isol8.config.json and container host load
  • noisy results: increase iterations and run with minimal background system activity

Performance tuning

Pool and concurrency tuning guidance for better latency.

Server overview

How isol8 serve applies pooling defaults for remote execution.

Configuration reference

Set poolStrategy, poolSize, and other execution defaults.

Architecture

Understand execution path and pool lifecycle internals.