- 01Request arrivesNo prepared clone is assigned
- 02Fresh microVM restoresPrivate writable filesystem
- 03CDP is returnedReady for one customer
We ran 10,000 isolated browser sessions on one server
We found a 1.37-second file copy around a 4-millisecond VM restore, then ran 10,000 isolated browser sessions on one server at 110.3/s with zero observed loss.
Firecracker loaded our snapshot in about 4 ms. Rookery then spent an average of 1.37 seconds copying 977 MiB of filesystem data before the browser could use it.
We had built a fast VM restore inside a slow browser restore path.
That discovery started a sequence of fixes, rejected experiments, and repeated benchmarks. The final run completed 10,000 out of 10,000 isolated browser sessions on one server at 110.3 sessions per second. Checkout reached 109 ms at p50 and 259 ms at p99. The complete service lifecycle reached 421 ms at p50 and 645 ms at p99.
This is the story of how the bottleneck moved. It is also a benchmark report, so the workload and clock boundaries appear before the final result. The test includes a CDP connection, proxied navigation, rendered-content extraction, and microVM teardown. It does not include a public edge or distributed control plane.
The constraint: destroy every browser
A browser session contains cookies, cache, local storage, downloads, and proxy state. It may also hold an authenticated customer session. Handing that browser to the next customer is not an acceptable pooling strategy.
Rookery gives every lease a fresh Firecracker microVM with its own guest kernel, filesystem, network, and browser process. Release destroys the Firecracker process and its writable filesystem.
This should be slow. Starting Linux and Chrome from zero for every request takes far longer than the scrape itself. Rookery instead restores a snapshot taken after Chrome is running, prepares the fresh browser, and returns a brokered Chrome DevTools Protocol URL.
"Cold" means Rookery creates the fresh clone after the request arrives. "Prepared" means it creates that clone before the request. Both paths end with a new microVM assigned to one customer, then destroyed.
Cold and prepared sessions keep the same isolation
- 01Fresh microVM restoresCreated before demand arrives
- 02Clone waits readyIsolated in the prepared pool
- 03CDP is returnedReady for one customer
The VM was waiting on a file copy
Our first deployment stored the snapshot artifacts on a filesystem without reflink support. A reflink is a copy-on-write clone: the new file initially shares unchanged disk blocks with its source and allocates private blocks only when it writes.
Without that capability, every restore copied roughly 977 MiB of allocated root filesystem data. At restore concurrency 16, the rootfs clone averaged 1,368 ms while Firecracker's snapshot load remained about 4 ms. A 60-worker test sustained 12.90 sessions per second.
Four milliseconds inside Firecracker, 1.37 seconds around it
Moving the mutable artifacts to reflink-capable storage removed the full copy from the hot path without weakening the writable-filesystem boundary. In the final 10,000-session run, the clone phase averages 13 ms. The complete snapshot restore boundary averages 47 ms, and the inclusive boundary from restore start until the VM is ready averages 74 ms.
Firecracker was doing what we asked. We were measuring the product boundary around it too coarsely to see where the time went.
The optimization from the research made it slower
The next candidate looked unusually convincing. Kernel describes using
userfaultfd to serve snapshot
memory on demand, while Browser Use reports
that larger memory pages reduced expensive exits in its nested-virtualization
stack.
We built an opt-in userfaultfd backend, produced a hugepage snapshot, and
tested lazy fault handling plus full fault-ahead prefetch on the EPYC host. The
existing File backend restored in 735 ms in that early matched test. Lazy
userfaultfd took 899 ms, and prefetch took roughly 1.1 seconds.
The File backend already used efficient in-kernel lazy fault-in from a warm page cache. Our workload did not have the 4 KiB page-fault storm the experiment was designed to remove, so moving faults into a userspace handler added work.
Those absolute times predate the storage migration and are not comparable to the final 47 ms restore mean. The ordering inside the matched test was clear, though. We kept the code for a future workload and left the production File path alone.
One extra page survived into every clone
Our next snapshot held two Chrome pages: the page prepared for proxied work and an unused default page. Every restored VM closed one renderer before it could attach to the other. That looked trivial until we measured it.
We built a snapshot with exactly one blank page in the prepared proxy context, then ran five 1,000-session tests per artifact in counterbalanced order with the same binary. Every one of the 10,000 measured sessions completed.
| Metric | Two-page snapshot | One-page snapshot |
|---|---|---|
| Throughput median | 82.46/s | 97.82/s |
| Checkout p50 / p99 | 288 / 482 ms | 177 / 363 ms |
| Service p50 / p99 | 732 / 986 ms | 595 / 830 ms |
| Control preparation mean | 152 ms | 33 ms |
The throughput change is (97.82 - 82.46) / 82.46 = 18.6%. The snapshot
restore itself did not improve. Moving one piece of browser cleanup before the
snapshot removed it from every future checkout.
Then a one-second tail kept appearing
Once storage and browser preparation were fast, small waits stopped looking small. Firecracker exposes its API through a Unix socket. Our old readiness gate checked for the socket pathname every 50 ms, which imposed a polling floor and could race with a path that existed before its listener accepted connections.
The replacement attempts a real connection every millisecond. A successful connection proves the API is accepting work. In the final sustained run, socket wait averages 14 ms and stays within 50 ms at p99.
That still left an occasional readiness stall of almost exactly one second. Packet tracing caught the host sending a TCP SYN while its virtual-link neighbor entry remained unresolved. Chrome was ready and the Firecracker vCPU was scheduled; the packet was waiting on address resolution. Stable link identities and neighbor entries removed the repeated stall in the matched startup trace. The final 10,000-session run recorded no slow-readiness logs.
Restore concurrency followed the same pattern. Raising the limit eventually created storage and scheduling contention. Repeated matched-load sweeps selected 12 simultaneous restores for the final configuration.
What one benchmark session includes
The workload runs against the production Rookery service on one Hetzner server. The client runs on that same host, so the numbers include the local HTTP service but exclude public Internet latency and a distributed control plane.
| Component | Benchmark configuration |
|---|---|
| Host | One AMD EPYC 9454P server, 48 physical cores and 251 GiB RAM |
| Guest | 2 vCPU, 2 GiB RAM, with Chrome already running in the snapshot |
| Startup check | One 60-request burst before the sustained workload |
| Runtime limits | 12 simultaneous restores and at most 60 active leases |
| Sustained load | 111 offered sessions/s with 60 workers |
| Workload | 10,000 sessions with CDP, proxy, navigation, extraction, and release |
| Population | 5,747 cold sessions and 4,253 prepared sessions |
The open-loop scheduler offers work at 111 sessions per second. Once all 60 leases are active, later arrivals wait in the client scheduler. The benchmark therefore tests 10,000 total sessions under sustained offered load, not 10,000 simultaneous browsers.
10,000 total sessions, with at most 60 active
- offered load111/sopen-loop arrivals
- capacity60maximum active leases
- completed10,000/10,000zero observed failures
- client wall
- 90.66 s
- successful throughput
- 110.3/s
- pool recovery
- <1 ms
10,000 total does not mean 10,000 simultaneous browsers.
Checkout begins when the client requests a lease and ends when Rookery returns a usable CDP URL. The service lifecycle continues through CDP connection, target attachment, proxied navigation, rendered-content extraction, and release. The client-observed lifecycle also includes any scheduler queue before checkout.
The browser loads HTML, CSS, JavaScript, and an image through a restricted local proxy. The harness subscribes to Chrome's page-load event, verifies rendered content, and then releases the VM. It does not retry failed attempts or build percentiles from successful retries.
10,000 sessions complete without observed loss
The sustained workload completes 10,000 out of 10,000 sessions in 90.66 seconds. Recovery-inclusive throughput is 110.3 successful sessions per second. Peak usage reaches the configured limit of 60 simultaneous leases.
| Phase | p50 | p95 | p99 | Maximum |
|---|---|---|---|---|
| Checkout | 109 ms | 194 ms | 259 ms | 373 ms |
| CDP connect and target attach | 7 ms | 49 ms | 55 ms | 79 ms |
| Navigation and load | 153 ms | 187 ms | 205 ms | 256 ms |
| Rendered-content extraction | 44 ms | 48 ms | 51 ms | 63 ms |
| Release | 118 ms | 168 ms | 195 ms | 239 ms |
| Service lifecycle | 421 ms | 556 ms | 645 ms | 821 ms |
The checkout population matters because a prepared pool can make an aggregate result look faster. Cold sessions account for more than half of this run.
| Checkout path | Samples | p50 | p95 | p99 | Maximum |
|---|---|---|---|---|---|
| All sessions | 10,000 | 109 ms | 194 ms | 259 ms | 373 ms |
| Cold | 5,747 | 132 ms | 216 ms | 286 ms | 373 ms |
| Prepared | 4,253 | 70 ms | 82 ms | 91 ms | 121 ms |
The separate startup burst drains the five prepared sessions and creates 55 cold sessions at once. Those 55 cold checkouts reach 163 ms at p50 and 263 ms at maximum. We treat that small burst as a startup diagnostic rather than using its p99 as a headline result.
Queueing changes what the user observes
Service latency and client-observed latency answer different questions. The service lifecycle measures work from checkout through release. The observed lifecycle starts at the scheduled arrival time and includes client queueing when the 60-lease limit is full.
One session, three different clocks
- 01Scheduler queuebefore checkout0.23 / 193 ms
- 02Checkoutusable CDP URL109 / 259 ms
- 03CDP attachconnect + target7 / 55 ms
- 04Page worknavigate + extract153 / 205 ms nav
- 05Releasedestroy session118 / 195 ms
| Boundary | p50 | p95 | p99 | Maximum |
|---|---|---|---|---|
| Client scheduler queue | 0.23 ms | 35 ms | 193 ms | 272 ms |
| Service lifecycle | 421 ms | 556 ms | 645 ms | 821 ms |
| Client-observed lifecycle | 422 ms | 609 ms | 744 ms | 888 ms |
Both paths remain below one second at p99 in this run. Client-side queueing still widens the observed tail once active leases reach the configured capacity. Reporting both prevents server-side processing time from being presented as end-user latency.
The tempting comparison is also the wrong one
Browser Use reports 825 ms p50 and 1.35 s p99 browser creation through its public API during a 10,000-session stress test. It also reports that every browser started and that Chromium startup after resume takes roughly 545 ms at p50.
We built Rookery, so we have every incentive to choose the boundary that makes it look fastest. Our closest boundary is cold checkout, but these tests are not controls for one another.
| Difference | Browser Use | Rookery |
|---|---|---|
| Closest create-ready p50 | 825 ms | 132 ms cold checkout |
| Closest create-ready p99 | 1,350 ms | 286 ms cold checkout |
| Client path | Public API and distributed fleet | Host-local API on one server |
| Snapshot timing | Before Chromium starts | After Chrome is already running |
| Disclosed load | Not stated | 111 offered sessions/s, capped at 60 leases |
The arithmetic is 825 / 132 = 6.3 at p50 and 1,350 / 286 = 4.7 at p99.
Those ratios compare published boundaries, not identical systems. Browser Use
includes its public edge and distributed control plane, while our client shares
a host with Rookery. Snapshot timing also moves a large piece of Chromium
startup outside our restore path.
Browser Use proves fleet placement, public connectivity, and a published browser-hour price. This single-host benchmark proves none of those things. We have not completed equivalent public cost accounting, so this article makes no price comparison.
Honest limits
- One 10,000-session run shows the observed distribution, not repeatability across days, software versions, or hosts.
- The workload tests 10,000 total sessions with at most 60 active leases, not 10,000 concurrent browsers or multi-host autoscaling.
- The host-local client excludes public routing, TLS, edge authentication, and a remote control plane.
- The controlled page and proxy validate browser behavior and request routing. They do not test residential egress, CAPTCHA solving, anti-bot success, or the wider Internet.
- The run uses a clean committed build. The harness records its source revision, unmodified-worktree marker, and binary checksum with the result.
- We retain raw attempt data and checksums, but have not published them as an immutable archive. Readers cannot independently audit every attempt yet.
Our separate posts explain when a page needs a browser and when to enable stealth mode. Those are product decisions outside this infrastructure benchmark.
A fast VM is only one boundary
Rookery does not reach 110.3 sessions per second by sharing browsers between customers. It restores fresh microVMs from a snapshot taken at the right point, avoids copying the entire filesystem, and keeps readiness and teardown work off the long tail.
The 10,000-session benchmark shows those choices holding under sustained load on one server. Cold checkout reaches 132 ms at p50, and the complete controlled service lifecycle remains below one second at p99.
That is the useful result for the browser path in webscrape.ai: strong session isolation can coexist with browser startup measured in hundreds of milliseconds, provided the benchmark keeps capacity and queueing visible.
If you benchmark your own browser runtime, start the clock before checkout and stop it after teardown. Users wait for the session, not the VM call.