Our pick for the part that actually costs money · Verified 18 August 2026
Thordata — pick Playwright for new work, then pay for access rather than for the library
SERP API $0.70–$0.80 per 1,000 queries; Web Scraper API $0.50–$1.00 per 1,000; Web Unlocker $1.00 per 1,000; Scraping Browser $2.50/GB; residential from $2.00 for 1GB, falling to $1.00/GB at 150GB
For new collection work, use Playwright: auto-waiting removes the largest single source of flaky runs, one API covers Chromium, Firefox and WebKit, and the Python and .NET bindings are first-party rather than community-maintained. That decision is free. What is not free is staying able to load the page at all — and once a target starts fingerprinting or rate-limiting, neither Playwright nor Selenium has an answer, because neither is in the access business. Thordata is: a residential pool spanning 100M+ IPs across 190+ countries, plus managed endpoints that hand back parsed results instead of a browser you have to keep alive. Route Playwright through residential proxies when you need your own browser logic, and drop to the SERP or Web Scraper API when you do not — a search-results harvest at $0.70–$0.80 per 1,000 queries costs less than the engineer-hours you would spend nursing a self-run browser fleet through the same volume. Prices verified 18 August 2026.
Verified 18 August 2026 Opens on the vendor’s site · CASRAI referral link
Not sure you need a browser at all? → — Most research collection fails on IP reputation, not on rendering. That page frames the access layer before you commit to any library.
Editorial disclosure: CASRAI has commercial referral arrangements with some of the vendors named on this page, and may earn a commission if you subscribe to them. We name them here regardless of whether a link is present. We only recommend tools our editorial team has independently researched. Read our full disclosure policy.
In summary
- Playwright wins for new automated collection: auto-waiting, one API across Chromium, Firefox and WebKit, and first-party Python, .NET and Java bindings.
- Selenium wins on institutional reality — WebDriver is a W3C standard, the ecosystem is two decades deep, and an existing Grid plus years of tests is not worth rewriting for its own sake.
- Puppeteer is the third tab you were about to open: excellent, Chrome-family only, JavaScript-first, and largely superseded by Playwright for cross-browser work.
- Neither library solves being blocked. Thordata: residential from $2.00 for 1GB down to $0.65–$0.73/GB at 1,000GB+, SERP API $0.70–$0.80/1K, Web Scraper API $0.50–$1.00/1K, Web Unlocker $1.00/1K, Scraping Browser $2.50/GB. Verified 18 August 2026.
- Do not migrate an existing Selenium Grid if your pain is blocking rather than flakiness — you will spend a quarter rewriting selectors and get blocked at exactly the same rate.
The differences that actually change your collection run
Thordata is the only priced product here; we publish no prices for open-source projects or for vendors whose pages we have not read ourselves
| Dimension | Playwright | Selenium | Puppeteer | Thordata (access layer) |
|---|---|---|---|---|
| What it is | Browser automation library driving browsers over their own protocols | W3C WebDriver implementation plus Grid for distributed runs | Chrome-family automation library, JavaScript-first | Proxy network and managed scraping APIs — not a browser library |
| Waiting and flakiness | Auto-waiting on actionability by default; the single biggest reliability gain | Explicit waits you write and maintain yourself | Waiting helpers, but more of it is on you than in Playwright | Not applicable — but a blocked request looks exactly like a flaky one in your logs |
| Browser coverage | Chromium, Firefox and WebKit through one API | Widest real-browser coverage including Safari and Edge via their own drivers | Chrome and Chromium; Firefox support is secondary | Scraping Browser at $2.50/GB when you want the browser hosted for you |
| Language bindings | First-party JavaScript/TypeScript, Python, Java and .NET | The broadest ecosystem in the category, including Ruby and long-tail community bindings | JavaScript/TypeScript; Python ports are community-maintained | Language-agnostic — HTTP endpoints and standard proxy configuration |
| Scaling to volume | Browser contexts are cheap; you still run and pay for the infrastructure | Grid is mature and battle-tested, and a real operational burden to run | Same infrastructure burden, narrower browser story | SERP API $0.70–$0.80/1K and Web Scraper API $0.50–$1.00/1K remove the fleet entirely |
| When the target starts blocking | No answer — it is not what the project is for | No answer, and the default driver profile is more conspicuous | No answer | Residential routing across 190+ countries; Web Unlocker at $1.00/1K for the hard pages |
Playwright, Selenium and Puppeteer are open-source and free to use; your cost is infrastructure and engineering time. Thordata prices are read off the vendor’s own pricing page and verified 18 August 2026. We do not print prices for any vendor whose page we have not read ourselves.
Playwright for new work, Selenium if you already have a Grid
Start with the decision, because you have read enough neutral surveys. For a new automated collection project at a university, institute or evidence-synthesis unit, choose Playwright. The reason is not feature count. It is that Playwright’s auto-waiting removes the single largest cause of runs failing overnight, and long unattended collection runs are precisely what research data work consists of. Add first-party Python bindings — which matters when the person maintaining the harvester is a research software engineer, not a QA specialist — and the choice is straightforward.
Choose Selenium when the institution already has it. If you run a Grid, have colleagues who know it, and have accumulated years of working selectors and page objects, the correct decision is almost always to keep it and spend the saved quarter on the access layer instead. Selenium also holds a genuine advantage on breadth: WebDriver is a W3C standard rather than one company’s protocol, real Safari and real Edge are supported through their vendors’ own drivers, and the binding ecosystem reaches languages Playwright has never targeted. If your collection has to be reproducible in a specific named browser build for a methods section, that standardisation is worth something.
What neither answer changes is the failure mode. Research collection at volume does not usually break because a library waited badly. It breaks because the target site decided you were a bot on request 4,000 of 60,000, and from then on you were harvesting error pages that your parser cheerfully recorded as empty results.
Auto-waiting, and why flaky runs are worse in research than in QA
Selenium asks you to reason about time. You locate an element, and if the page has not finished rendering it, you get an exception — so you write explicit waits, then longer explicit waits, and eventually a sprinkling of fixed sleeps that make the suite slow instead of broken. Playwright inverts this: actions wait for the element to be attached, visible, stable and able to receive events before they fire, and the wait is bounded by a timeout rather than by your guesswork. In practice this collapses a large class of intermittent failures that experienced Selenium users have simply stopped noticing they work around.
Two more Playwright details earn their place in a collection context. Browser contexts give you isolated, cheap sessions inside one browser process, which is the right primitive when you are harvesting hundreds of separate targets and do not want cookie state bleeding between them. And trace viewing — a recorded timeline with DOM snapshots and network activity — turns “the Tuesday night run produced 300 blank records” from an archaeology project into a five-minute inspection.
Why this matters more for research than for testing: a flaky test fails loudly and someone re-runs it. A flaky harvester succeeds quietly and writes incomplete data into your dataset. Nobody re-runs it, because nothing looked wrong. Months later a reviewer asks about coverage and you cannot answer. The habit that protects you is not library choice at all — it is asserting on the shape of what you collected, so a run that returns 40% of the expected records fails loudly rather than committing silently.
Browser coverage, bindings and the standards argument
Concede this properly, because it is the honest half of the comparison. Selenium implements the W3C WebDriver specification, which means it drives browsers through an interface the browser vendors themselves committed to. Playwright drives Chromium, Firefox and WebKit through lower-level protocols with patched browser builds it ships and manages. That is why Playwright feels faster and more capable — and also why “WebKit” in Playwright is not literally the Safari your participants use, even though it is the same rendering engine family.
If your protocol says the collection ran in Safari 18 on macOS, Selenium with Apple’s safaridriver is the answer that survives a methods review. The same applies to genuine Microsoft Edge, to older browser builds a regulated environment has pinned, and to any situation where an auditor may ask which browser produced a record.
On bindings, Selenium’s reach is broader — Ruby and a long tail of community languages included — but Playwright’s coverage of the four that matter to most research teams is first-party and kept in step across releases. If your group is Python-centred, the practical difference is small and Playwright’s async API is the more pleasant one to build a pipeline around.
The remaining Selenium advantage is institutional rather than technical: Grid is a known quantity, your IT department may already run it, and two decades of Stack Overflow answers exist for every error message it can emit. Do not dismiss that. A tool your colleagues can already debug is worth more than a marginally better API nobody else in the building understands.
Puppeteer vs Playwright, settled quickly
Puppeteer is the next search most people make after this one, so here it is. Puppeteer is a well-built, well-documented library for driving Chrome and Chromium from JavaScript. It came first, Playwright was built by people who had worked on it, and the family resemblance in the APIs is obvious the moment you read both.
The differences that decide it: Playwright covers WebKit and Firefox as first-class targets while Puppeteer is Chrome-family with secondary Firefox support; Playwright ships first-party Python, Java and .NET bindings while Puppeteer is JavaScript-first with community ports; and Playwright’s auto-waiting, contexts and tracing are more developed. For a purely Chrome-only Node.js job, Puppeteer remains a perfectly rational choice with a slightly smaller surface area to learn. For anything cross-browser, or anything your Python-writing colleagues will inherit, Playwright is the better bet.
One thing all three share, and it is the thing this page keeps returning to: none of them changes how a target site perceives your traffic. Swapping Puppeteer for Playwright to get past a block is a category error. The request still leaves from the same IP address, at the same rate, with the same fingerprint characteristics that got you blocked in the first place.
What migrating off a Selenium Grid really costs
Vendor comparisons treat migration as a syntax exercise. It is not, and a research group with one engineer should hear the real shape of it before committing a term’s work.
- Selectors and page objects. Playwright’s locator model is better, which also means it is different. Mechanical translation of a mature Selenium suite produces code that works but throws away the reliability you migrated for, so the honest budget is a rewrite of the interaction layer rather than a find-and-replace.
- Infrastructure. Grid nodes, your CI runners, and any container images built around WebDriver all change. If your Grid is managed centrally by IT rather than by your group, add the lead time for that conversation — it is frequently the longest item on the list.
- Institutional knowledge. The colleague who can fix the harvester at short notice knows Selenium. After migration, they do not. Cross-training is a real line in the budget.
- Anything not the browser. Authentication flows against your institutional SSO, VPN-only sources, licensed database terms, and the scheduling that runs it all — none of this transfers automatically.
A sane middle path exists and most groups should take it: leave the existing Grid alone, and write new collectors in Playwright. The two coexist without difficulty. Migrate an old collector only when you are already opening it for other reasons.
Do not migrate at all if your actual complaint is that runs return empty pages, CAPTCHAs or 403s. That is a blocking problem wearing a flakiness costume, and a full rewrite will land you in exactly the same place a quarter later with a smaller budget. Fix the access layer first; if the numbers then improve, you have your answer about what was really broken.
Neither library gets you through a site that has decided to stop you
Here is what actually happens on a collection run at research volume. The first few hundred requests succeed. Then response times lengthen, then an interstitial appears, then you are served a challenge page, and finally the responses turn into 403s. Nothing in your Playwright or Selenium code changed. The site’s edge simply concluded that a single IP address requesting structured pages at machine cadence was not a person, which — to be fair — it was not.
No amount of library tuning addresses this, because the library was never the signal. The signals are the IP address and its reputation, the request cadence, TLS and header characteristics, and behavioural patterns across a session. Two of those are network-layer properties that live entirely outside your automation code.
This is where the access layer earns its keep, and where Thordata enters our recommendation. Two patterns cover nearly all research collection:
- Keep your own browser, change the exit point. Route Playwright or Selenium through residential proxies so requests leave from ordinary consumer addresses across a pool of 100M+ IPs in 190+ countries. Residential traffic starts at $2.00 for 1GB and falls with commitment — $1.80/GB at 10GB, $1.50/GB at 50GB, $1.00/GB at 150GB, $0.80/GB at 350GB and $0.65–$0.73/GB from 1,000GB. Geography matters more than teams expect: a national funder registry or a country-specific pricing page frequently serves different content by region, and choosing the exit country is part of the methodology.
- Stop running a browser at all. If what you need is search results or structured page data rather than bespoke interaction, a managed endpoint returns parsed output and absorbs the blocking problem as someone else’s operational concern — SERP API at $0.70–$0.80 per 1,000 queries, Web Scraper API at $0.50–$1.00 per 1,000, Web Unlocker at $1.00 per 1,000 for pages that fight back, and a hosted Scraping Browser at $2.50/GB when you genuinely need a real browser but not a fleet to maintain. All figures verified 18 August 2026.
The honest trade-off: this is a recurring cost where the library was free, and it introduces a dependency your data-management plan should name. In exchange you stop paying an engineer to babysit a browser farm, and your collection finishes. Weigh it against the true alternative, which is not “free” — it is unpaid staff time plus incomplete datasets. Whatever you decide, keep the ethical side straight: honour the terms of licensed databases, prefer an official API where one exists, rate-limit out of courtesy rather than only out of necessity, and get your ethics or legal review done before the run rather than after. Our guide to web scraping proxies works through proxy types and compliance in more depth, and competitive intelligence tools covers the case where you would rather buy the finished dataset than collect it.
Price the access layer before you rewrite a line of code
Estimate the bandwidth or the query count of one full collection run, then check it against the actual figures: residential from $2.00 for 1GB down to $0.65–$0.73/GB at 1,000GB+, SERP API $0.70–$0.80 per 1,000 queries, Web Scraper API $0.50–$1.00 per 1,000, Web Unlocker $1.00 per 1,000, Scraping Browser $2.50/GB. Verified 18 August 2026. Most research runs are far smaller than teams assume, and the number is usually less than a week of the engineering time currently spent restarting failed harvests.
Residential from $0.65/GB at volume
See Thordata pricing Opens on the vendor’s site · CASRAI referral link
Frequently asked questions
Playwright vs Selenium — which is better for automated research data collection?
Playwright, for new work. Auto-waiting eliminates the most common cause of overnight runs failing, browser contexts give you cheap session isolation across many targets, and the trace viewer turns an incomplete dataset into a diagnosable event rather than an archaeology project. First-party Python bindings matter if a research software engineer rather than a QA specialist maintains the collector. Choose Selenium instead when you already run a Grid, when colleagues know it, or when your protocol requires a specific named browser such as real Safari — WebDriver is a W3C standard and drives vendors’ own drivers, which is the stronger position for reproducibility claims.
Is Playwright faster than Selenium?
Usually yes, for two structural reasons rather than one clever optimisation. Playwright talks to browsers over their own protocols instead of through the WebDriver layer, and its auto-waiting removes the fixed sleeps that people scatter through Selenium suites to make them stable. In a collection pipeline the more valuable gain is not raw speed but variance: fewer runs that take three times as long or fail halfway. Beyond a few concurrent browsers, though, your throughput is set by the target site’s tolerance and your network path, not by the library — which is exactly when the access layer becomes the limiting factor.
Puppeteer vs Playwright — is there still a reason to pick Puppeteer?
Yes, narrowly. If the job is Chrome-only, written in Node.js, and you value a smaller API surface, Puppeteer is mature, well documented and entirely adequate. Playwright wins whenever you need WebKit or Firefox as first-class targets, whenever Python, Java or .NET colleagues will maintain the code, or whenever you want the more developed auto-waiting, contexts and tracing. Neither choice affects whether a target site blocks you, so do not switch between them hoping to get past a challenge page — the request leaves from the same address either way.
Will switching to Playwright stop us getting blocked?
No, and any comparison that implies otherwise is selling something. Blocking decisions are made on IP reputation, request cadence, network-layer fingerprint and session behaviour. The first two of those are not properties of your automation library at all. Switching libraries changes the code that drives the browser; it does not change where the traffic comes from or how fast it arrives. If empty pages, interstitials and 403s are your symptom, spend the budget on routing and rate discipline rather than on a rewrite — you will find out within days whether that was the real problem.
What does it cost to run collection through a proxy or scraping API?
For Thordata, residential traffic starts at $2.00 for 1GB and falls with volume — $1.80/GB at 10GB, $1.50/GB at 50GB, $1.00/GB at 150GB, $0.80/GB at 350GB, and $0.65–$0.73/GB from 1,000GB. Datacentre and static ISP addresses are $0.75/IP and mobile is $2.20/GB. If you would rather not run a browser at all, the SERP API is $0.70–$0.80 per 1,000 queries, the Web Scraper API $0.50–$1.00 per 1,000, Web Unlocker $1.00 per 1,000, and the hosted Scraping Browser $2.50/GB. All verified 18 August 2026. Estimate one run before you commit to a tier — research collections are usually smaller than teams expect, and the entry point is deliberately low enough to test with real targets. Our web scraping proxies guide compares the routing options in detail.
We have a working Selenium Grid. Is migrating worth it?
Probably not as a project in its own right. The real cost is not syntax translation — it is rewriting the interaction layer to actually use Playwright’s locator model, rebuilding CI images and Grid infrastructure, retraining the colleague who fixes things at short notice, and re-testing every SSO and VPN path your collectors depend on. The pragmatic route is to leave the Grid running, write new collectors in Playwright, and migrate old ones only when you are opening them anyway. Do not migrate at all if your complaint is blocked requests rather than flaky waits: that is an access problem, and a rewrite will reproduce it faithfully.
When should we avoid a headless browser entirely?
Whenever something cheaper returns the same data. If the source publishes an API, use it — it is faster, more stable, and defensible in a methods section. If plain HTTP requests return the content because the page is server-rendered, a browser adds bandwidth and fragility for nothing. If you need search results or standard structured pages at volume, a managed scraping endpoint returns parsed output without you operating any browser at all. Keep Playwright or Selenium for what genuinely requires them: authenticated sessions, multi-step interaction, and content that only exists after client-side rendering.
