Set up Browser Use with an AI agent

name: browser
description: Use when building AI browser automation agents, controlling cloud
  or local browsers, extracting web data, filling forms, testing websites, or
  automating multi-step web workflows. Choose between open-source Python library
  (local/self-hosted) or Cloud API (managed infrastructure with stealth
  browsers, CAPTCHA handling, residential proxies).
metadata:
  mintlify-proj: browser
  version: "1.0"

Browser Use Skill

Product summary

Browser Use is an AI browser automation framework with two deployment paths: open-source Python library for local/self-hosted agents, and Cloud API for managed infrastructure. The open-source library (browser-use package) lets you connect any LLM (Claude, GPT, Gemini, etc.) to control a browser programmatically. The Cloud API (REST v4 + SDKs) runs agents on managed infrastructure with stealth browsers, CAPTCHA solving, residential proxies, and persistent sessions.

Key files & commands:

• Open-source: Agent(task=..., llm=..., browser=...) with await agent.run()
• Cloud: client.runs.create(task=...) or client.browsers.create() for infrastructure
• Environment: .env file with BROWSER_USE_API_KEY or LLM provider keys
• Primary docs: https://docs.browser-use.com

When to use

Reach for Browser Use when:

• Building AI agents that need to interact with websites (form filling, data extraction, navigation, testing)
• Automating multi-step workflows that require visual understanding and DOM interaction
• Extracting structured data from websites that don't have APIs
• Testing web applications with QA workflows
• Handling authentication (login, 2FA, cookies, profiles)
• Running in production with stealth, CAPTCHA handling, or residential proxies (use Cloud)
• Developing locally with full control over browser and LLM (use open-source)
• Integrating with coding agents (Claude, Cursor, Codex) via MCP server or CLI

Do NOT use Browser Use for:

• Simple HTTP requests (use requests/httpx instead)
• JavaScript-heavy SPAs where you only need data (consider Playwright directly)
• Tasks that don't require visual grounding or complex interaction

Quick reference

Open-Source: Core Agent Setup

from browser_use import Agent, ChatBrowserUse
import asyncio

async def main():
    agent = Agent(
        task="Find the top Hacker News story",
        llm=ChatBrowserUse(),  # or ChatOpenAI, ChatAnthropic, etc.
    )
    result = await agent.run()
    print(result.extracted_content)

asyncio.run(main())

Cloud: Run a Task

from browser_use_sdk.v4 import BrowserUse

with BrowserUse() as client:
    run = client.runs.create("Find the top Hacker News story")
    result = client.runs.wait_for_completion(run.id)
    print(result.result)

Essential Agent Parameters

Parameter	Type	Purpose
task	str	Natural language instruction (1-50k chars)
llm	LLM object	Model to use (ChatBrowserUse, ChatOpenAI, ChatAnthropic, etc.)
browser	Browser	Browser config (headless, viewport, proxy, etc.)
max_steps	int	Max actions before stopping (default: 100)
output_model_schema	Pydantic	Structured output validation
allowed_domains	list	Restrict navigation to specific domains
start_url	str	Initial page (saves steps)
keep_alive	bool	Keep browser open after task (Cloud only)

Browser Configuration (Open-Source)

from browser_use import Browser

browser = Browser(
    headless=False,           # Show window
    window_size={'width': 1000, 'height': 700},
    proxy={'server': 'http://proxy:port'},
    storage_state='./auth.json',  # Load cookies/localStorage
)

Available Browser Tools

Tool	Purpose
navigate(url)	Go to URL
click(index)	Click element by index
type(text)	Type into focused field
scroll(direction, amount)	Scroll page
screenshot()	Capture page
extract(prompt)	Extract data with LLM
send_keys(keys)	Send keyboard shortcuts
upload_file(path)	Upload file to input
go_back()	Browser back button

Supported LLM Providers (15+)

Provider	Model Example	Env Variable
Browser Use	ChatBrowserUse()	BROWSER_USE_API_KEY
OpenAI	ChatOpenAI(model="gpt-5")	OPENAI_API_KEY
Anthropic	ChatAnthropic(model="claude-opus-5")	ANTHROPIC_API_KEY
Google	ChatGoogle(model="gemini-3-flash")	GOOGLE_API_KEY
DeepSeek	ChatDeepSeek(model="deepseek-chat")	DEEPSEEK_API_KEY
Groq	ChatGroq(model="llama-4-maverick")	GROQ_API_KEY

See supported-models for 15+ providers.

Decision guidance

When to use Cloud vs Open-Source

Scenario	Cloud	Open-Source
Production with stealth/CAPTCHA/proxies	✓	✗
Local development/testing	✗	✓
Full control over LLM	✗	✓
Managed infrastructure	✓	✗
Persistent sessions/profiles	✓	✓
Cost-sensitive (low volume)	✓	✓
High concurrency (100+ parallel)	✓	✗

When to use Vision vs No Vision

Condition	Use Vision	Disable
Complex visual layouts	✓
Text-heavy pages	✓
Performance critical		✓
Sensitive data on screen		✓
Structured forms only		✓

Set use_vision=True (default) or False in Agent.

When to use Structured Output vs Free Text

Use Case	Structured	Free Text
Extract specific fields	✓
Validate output format	✓
Integration with APIs	✓
Exploratory/open-ended		✓
Human review needed		✓

Pass output_model_schema=MyPydanticModel for structured output.

Workflow

Typical Task Execution (Open-Source)

1. Set up environment

   • Create .env with BROWSER_USE_API_KEY or LLM provider key
   • Install: pip install browser-use
   • Install browser: uvx browser-use install

2. Define the task clearly

   • Be specific: "Go to example.com, fill form with name='John', submit, extract result"
   • Avoid vague: "Do something on the web"
   • Include step numbers if multi-step
   • Reference action names directly (click, type, extract, etc.)

3. Configure agent

   • Choose LLM (ChatBrowserUse recommended for speed/cost)
   • Set browser options (headless, viewport, proxy)
   • Add authentication if needed (storage_state, secrets, profile)
   • Set max_steps based on task complexity (default 100)

4. Run and monitor

   • Call await agent.run()
   • Check result.extracted_content or result.final_answer()
   • Review step history if debugging
   • Adjust task description if agent gets stuck

5. Verify output

   • Check structured output matches schema (if used)
   • Validate extracted data against expected format
   • Test with multiple runs for consistency
   • Review cost/token usage

Typical Cloud Task (API v4)

1. Create run → client.runs.create(task="...", model="...")
2. Poll status → client.runs.get_run_status(run.id)
3. Wait completion → client.runs.wait_for_completion(run.id)
4. Extract result → run.result or run.output (if structured)
5. Clean up → Delete session if not reusing

Multi-Step Workflow (Keep-Alive Session)

1. Create session with keep_alive=True (Cloud) or @sandbox decorator (open-source)
2. Send first task
3. Get live_view_url for human interaction if needed
4. Send follow-up tasks to same session (reuses browser state)
5. Close session when done

Common gotchas

• Tool parameter names must match exactly: Use browser_session: BrowserSession in custom tools, not browser: Browser. Mismatched names cause silent failures.

• Vision mode adds latency: Screenshots are sent to LLM on every step. Disable with use_vision=False if not needed.

• Timeouts on slow networks: Increase TIMEOUT_NavigateToUrlEvent, TIMEOUT_TypeTextEvent environment variables for slow connections.

• Element indices change after page updates: After clicking/typing, element indices shift. Agent handles this, but avoid hardcoding indices in custom tools.

• Passwords in prompts leak: Never put credentials in task description. Use secrets dict, storage_state, or @sandbox profile sync instead.

• CAPTCHA solving not guaranteed: Cloud CAPTCHA handling works for common types but may fail. Have fallback (human-in-the-loop, different proxy country).

• Disconnecting CDP doesn't stop browser: In Cloud, browser.close() or Playwright disconnect only closes your connection. Call client.browsers.stop(session.id) to actually stop the managed browser.

• Session browser stops after 2 minutes idle: If follow-up task arrives later, it gets a fresh browser. Set keep_alive (seconds) to hold browser open longer.

• Free-tier model restrictions: Free accounts can only use gpt-6-luna, deepseek-v4.1-flash, gemini-3-flash, minimax-m3, bu-ultrafast, bu-fast. Other models return 403.

• Allowed domains don't block subdomains by default: Use *.example.com pattern to match subdomains. Wildcards in TLD (e.g., example.*) are not allowed.

• Custom tools with wrong parameter names fail silently: The agent won't call your tool if parameter names don't match. Check browser_session, agent, llm parameter names carefully.

• Flash mode disables thinking: flash_mode=True skips reasoning steps for speed but may reduce accuracy. Use for simple tasks only.

• Structured output validation happens at end: If agent can't produce output matching schema, run fails with "Could not produce output matching outputSchema". Validate schema is achievable.

Verification checklist

Before submitting work with Browser Use:

• Task description is specific: Include step numbers, action names (click, type, extract), and expected output format
• LLM is configured: API key set in .env or environment, model is supported
• Browser config is correct: Headless mode, viewport, proxy settings match requirements
• Authentication is handled: Credentials in secrets dict or storage_state, not in task prompt
• Allowed domains set (if needed): Restrict navigation to prevent agent wandering
• Vision mode appropriate: Disabled if sensitive data on screen or performance critical
• Structured output schema valid: If using output_model_schema, schema is achievable and validated
• Max steps sufficient: Task complexity doesn't exceed max_steps (default 100)
• Error handling in place: Fallback LLM, retry logic, or human-in-the-loop for critical tasks
• Cost tracking enabled: calculate_cost=True if billing matters
• Session cleanup: Delete or purge sessions when done (Cloud)
• Live URL secured: If using keep_alive and human interaction, treat live_view_url as credential

Resources

• Comprehensive navigation: https://docs.browser-use.com/llms.txt — full page listing for agent reference
• Open-source quickstart: https://docs.browser-use.com/open-source/quickstart — install, configure, run first agent
• Cloud API v4 overview: https://docs.browser-use.com/cloud/api-v4-overview — REST endpoints, SDK, authentication
• Prompting guide: https://docs.browser-use.com/open-source/customize/agent/prompting-guide — write effective task descriptions

---

For additional documentation and navigation, see: https://docs.browser-use.com/llms.txt