POST /search/v2/scrape
Read one web page as Markdown
Reads one public http(s) page and returns the formats asked (markdown by default; html, rawHtml, links) with metadata. Galon reads as GalonBot, obeys robots.txt (RFC 9309: unreachable robots.txt means no read), sends at most one request per second per host and never reads private, local or reserved networks. No JavaScript rendering: waitFor and actions are refused (javascript_rendering_unavailable). A format Galon does not serve (screenshot, json, …) is refused by name (format_unsupported). One page is counted; timeout above 60000 ms is capped and said in `warning`.
API key (Bearer), on the public API endpoint.
Request body
application/json · schema CrawlScrapeRequest
Responses
| Status | Description | Body |
|---|---|---|
200 | The page, read | CrawlScrapeResponse |
400 | body_invalid, content_type_invalid, field_unknown, field_invalid, url_invalid, url_blocked, ipv6_unsupported, format_unsupported, option_unsupported, javascript_rendering_unavailable, robots_disallowed, robots_unreachable, page_too_large, content_type_unsupported, too_many_redirects, tls_invalid | CrawlError |
401 | api_key_missing, api_key_invalid | CrawlError |
402 | insufficient_balance (only when a price grid is set) | CrawlError |
403 | scope_missing (search:query) | CrawlError |
404 | not_found (service not open to this account) | CrawlError |
413 | body_too_large | CrawlError |
429 | rate_limited (Retry-After), daily_limit_reached, quota_exceeded, tenant_quota_exceeded | CrawlError |
502 | site_unreachable | CrawlError |
503 | search_unavailable, search_auth_unavailable, executor_unavailable, executor_result_invalid | CrawlError |
504 | scrape_timeout | CrawlError |
Examples
curl
curl -X POST "$GALON_API_BASE/search/v2/scrape" \
-H "Authorization: Bearer $GALON_API_KEY" \
-H "Content-Type: application/json" \
-d 'null'Node.js (SDK)
import { Galon } from "galon";
const galon = new Galon(process.env.GALON_API_KEY);
const result = await galon.search.scrape({});Python (SDK)
from galon import Galon
galon = Galon() # reads GALON_API_KEY
result = galon.search.scrape({})