My local Chrome fleet worked well until several target sites began applying stricter fingerprinting and automated-traffic checks. The same Playwright scripts now lose roughly half of their parallel workers, which detach or exit without useful logs. Pinning browser versions and slowing the pacing helped only slightly, and failures become much more common when three jobs run at once. I'm looking for a dependable setup that remains stable under load, whether that means a managed browser platform or a better way to operate local browsers. I'm mainly interested in completion rates, diagnostics, memory usage, and browser stability—not just avoiding individual checks.
4 Answers
Throttling local browsers can reduce a lot of noise, but it won’t fix every failure if a site changes its checks during a job. I’d separate the problem into two measurements: infrastructure failures such as OOMs, dead processes, and timeouts, versus legitimate access denials or challenge pages. Capture browser exit codes, page events, screenshots, and resource usage so you know which category is actually dominating.
The concurrency pattern sounds suspiciously like browser or node saturation. Selenium Grid and Kubernetes sidecars can look fine at two or three sessions, then start thrashing memory, restarting containers, and losing the trail once the load increases. Try fewer browsers per node, explicit memory limits, per-session logs, and a gradual load test. A lower worker count that completes reliably is usually better than a large pool that silently dies.
It may be worth testing a managed service such as Browserless or Browserbase while keeping your existing Playwright code. The main benefits are controlled concurrency, session diagnostics, and better visibility into browser exits. Test them against your actual sites and workloads rather than assuming a hosted service will solve everything. Also measure memory use and process health first, because failing at three workers could be resource exhaustion rather than fingerprinting alone.
Headed mode usually isn’t a dependable fix by itself. It can change resource usage and occasionally expose differences in rendering, but it won’t make an unstable fleet reliable. Focus on supported access methods, conservative concurrency, consistent browser environments, and clear retry rules instead of treating headed mode or proxy changes as a cure-all.
That matches my experience: switching modes sometimes changes the symptoms, but the dead-browser problem remains when the node is overloaded.