url | impl AsRef<str> | The URL to scrape. |
options.formats | Vec<Format> | Output formats: Markdown, Html, RawHtml, Links, Images, Screenshot, Summary, ChangeTracking, Json, Attributes, Branding, Product, Menu, Audio, Video, Question(QuestionFormat), Highlights(HighlightsFormat). |
options.headers | HashMap<String, String> | Custom HTTP headers. |
options.include_tags | Vec<String> | Only include content from these HTML tags. |
options.exclude_tags | Vec<String> | Exclude content from these HTML tags. |
options.only_main_content | bool | Strip nav, footer, and boilerplate. |
options.timeout | u32 | Timeout in milliseconds. |
options.wait_for | u32 | Wait for page to render (milliseconds). |
options.mobile | bool | Use a mobile viewport. |
options.parsers | Vec<ParserConfig> | File parsers. ParserConfig::Simple("pdf") or ParserConfig::Pdf { parser_type, mode?, max_pages?, pages?, blocks?, page_markers? }. |
options.actions | Vec<Action> | Pre-scrape browser actions: Wait, Click, Write, Press, Scroll, Screenshot, Scrape, ExecuteJavascript, Pdf. |
options.location | LocationConfig | { country, languages } for geo-aware scraping. |
options.skip_tls_verification | bool | Skip TLS verification. |
options.remove_base64_images | bool | Drop base64 images from markdown. |
options.fast_mode | bool | Faster scrapes with reduced fidelity. |
options.block_ads | bool | Block ads and cookie popups. |
options.proxy | ProxyType | Basic, Stealth, Enhanced, Auto. |
options.max_age | u32 | Accept cached data up to this age (milliseconds). |
options.min_age | u32 | Accept cached data only if at least this old (milliseconds). |
options.store_in_cache | bool | Cache the result. |
options.lockdown | bool | Serve only previously cached results; never make outbound requests. |
options.redact_pii | bool | Redact personally identifiable information. |
options.audit_metadata | AuditMetadata | User attribution for SIEM logging. Has field username. |
options.profile | ProfileConfig | Persistent browser profile: { name, save_changes? }. |
options.json_options | JsonOptions | JSON extraction: { schema?, system_prompt?, prompt?, check_prompt_injection? }. |
options.screenshot_options | ScreenshotOptions | Screenshot config: { full_page?, quality?, viewport? }. |
options.change_tracking_options | ChangeTrackingOptions | Change tracking: { modes? (GitDiff, Json), schema?, prompt?, tag? }. |
options.attribute_selectors | Vec<AttributeSelector> | Attribute extraction: { selector, attribute }. |