AI agents / Benchmark
We gave the same AI agent the same 50 public web tasks twice: once browsing through the Browz MCP server, once through a plain browser MCP. Here is how often each finished the task, how often a site stopped it, and what each run took in steps, tokens and time.
Measured, then published
tasks completed through Browz (38/50 through the plain browser)
tasks completed on sites that run bot defences (9/18 through the plain browser)
runs stopped by a block page or bot check through Browz (12 through the plain browser)
median time per task through Browz (23 s through the plain browser)
| Browz MCP | Plain browser MCP | |
|---|---|---|
| Tasks completed | 46 of 50 (92%) | 38 of 50 (76%) |
| …on sites with little bot protection | 32 of 32 (100%) | 29 of 32 (90.6%) |
| …on sites that run bot defences | 14 of 18 (77.8%) | 9 of 18 (50%) |
| Runs stopped by a block page, CAPTCHA or bot check | 3 of 50 | 12 of 50 |
| Median steps per task (tool calls) | 3 | 3 |
| Median tokens per task | 87,136 | 71,150 |
| Median time per task | 22.7 s | 23.3 s |
One recorded run of every task through each browser, on 2 November 2026, with Claude Sonnet. Medians are over all 50 tasks, finished or not.
Method
The tasks, the prompt and the scoring are fixed, so the run can be repeated and the two browsers get exactly the same work.
Short tasks of the kind real agents get, in the style of public web-agent benchmarks: look something up, search a site, read a price or a headline. 32 are on sites with little bot protection (an encyclopedia, a paper archive, code hosting, developer documentation, dictionaries, course catalogues, a news site) and 18 on well-known sites that run bot defences (search engines, online shops, travel, jobs, recipes, sports, film and housing sites).
No task signs in, signs up, buys anything or submits a form other than a site’s own search. The agent is told to stop if a site blocks it, not to work around it.
The same Claude model, the same prompt and the same limits (30 turns, 5 minutes) for each task: once with the Browz MCP server, once with the plain browser MCP server. The browser tools are the agent’s only way to the web; its own web search and fetch are switched off.
Through Browz, each task gets a new throwaway session: its own profile with a consistent device fingerprint, wiped when the task ends. Through the plain browser, each task gets a new isolated browser context. Both run headless, from the same computer and internet connection, side by side.
Each answer is checked automatically against the known answer or the expected format (a price, a title, a year), and only counts if the agent actually visited the site. A run counts as blocked when a block page, CAPTCHA or bot check stopped the agent.
Steps are the browser tool calls the agent made. Tokens are everything the model read and wrote for the task, cached input included, as the model provider reports them. Time is wall-clock time from the first prompt to the final answer.
FAQ
It is the most common way to give an agent a browser over MCP today: an open-source server that drives a standard browser. It is the setup most agents start with, so it is the fairest baseline.
Because that is where agents get stuck. A browser that looks automated is often shown a CAPTCHA or a block page instead of the content, and the task fails however good the model is.
Yes. Sites change their pages and their defences, and models improve. This is one recorded run on the date shown, and we will re-run it and publish the new numbers here.
Use agents only where the site’s terms and the law allow it, and on accounts you are entitled to use. Our acceptable use policy applies to agents the same as to people.