Galon Cloud
Galon Cloud

Galon Search

Read web pages and sites for your AI: scrape a page, map a site's links, crawl a site to Markdown.

Markdown · API reference · OpenAPI

Before you start

  • A Galon account, a project and a key with search:query.

Quickstart

Every example reads GALON_API_BASE (https://api.galon.ai/v1) and GALON_API_KEY from the environment and prints the HTTP status and the JSON body.

1. Set your key. Export the base URL and your key (console: Keys, scope search:query).

curl

export GALON_API_BASE="https://api.galon.ai/v1"
export GALON_API_KEY="gln_..."

2. Scrape a page. formats: markdown (default), html, rawHtml, links. JavaScript rendering is not offered (javascript_rendering_unavailable); timeout above 60000 ms is capped.

curl

curl -sS -X POST "$GALON_API_BASE/search/v2/scrape" \
  -H "Authorization: Bearer $GALON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "formats": ["markdown"]}' \
  -w '\n%{http_code}\n'

Python 3 (standard library)

import json, os, urllib.error, urllib.request

req = urllib.request.Request(
    os.environ["GALON_API_BASE"] + "/search/v2/scrape",
    data=json.dumps({"url": "https://example.com", "formats": ["markdown"]}).encode(),
    headers={"Authorization": "Bearer " + os.environ["GALON_API_KEY"],
             "Content-Type": "application/json"},
    method="POST",
)
try:
    with urllib.request.urlopen(req) as res:
        print(res.status, json.load(res))
except urllib.error.HTTPError as err:  # the error body carries detail.code and detail.next_action
    print(err.code, json.load(err))

Node.js 18+ (fetch)

const res = await fetch(`${process.env.GALON_API_BASE}/search/v2/scrape`, {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.GALON_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({url: "https://example.com", formats: ["markdown"]}),
});
console.log(res.status, JSON.stringify(await res.json()));

3. Map a site. The addresses of a site's pages (sitemaps and start page), without reading them; search puts the addresses that contain a text first.

curl

curl -sS -X POST "$GALON_API_BASE/search/v2/map" \
  -H "Authorization: Bearer $GALON_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "search": "pricing", "limit": 100}' \
  -w '\n%{http_code}\n'

Python 3 (standard library)

import json, os, urllib.error, urllib.request

req = urllib.request.Request(
    os.environ["GALON_API_BASE"] + "/search/v2/map",
    data=json.dumps({"url": "https://example.com", "search": "pricing", "limit": 100}).encode(),
    headers={"Authorization": "Bearer " + os.environ["GALON_API_KEY"],
             "Content-Type": "application/json"},
    method="POST",
)
try:
    with urllib.request.urlopen(req) as res:
        print(res.status, json.load(res))
except urllib.error.HTTPError as err:  # the error body carries detail.code and detail.next_action
    print(err.code, json.load(err))

Node.js 18+ (fetch)

const res = await fetch(`${process.env.GALON_API_BASE}/search/v2/map`, {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.GALON_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({url: "https://example.com", search: "pricing", limit: 100}),
});
console.log(res.status, JSON.stringify(await res.json()));

4. Crawl a site. A background job: limit pages (default 10, at most 100), maxDepth hops (at most 5), 15 minutes, 2 jobs at a time per project. Send an Idempotency-Key: the same key returns the same job, never a second one.

curl

curl -sS -X POST "$GALON_API_BASE/search/v2/crawl" \
  -H "Authorization: Bearer $GALON_API_KEY" \
  -H "Idempotency-Key: docs-2026-11-14" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com/docs", "limit": 25, "maxDepth": 2, "excludePaths": ["/docs/archive/.*"]}' \
  -w '\n%{http_code}\n'

Python 3 (standard library)

import json, os, urllib.error, urllib.request

req = urllib.request.Request(
    os.environ["GALON_API_BASE"] + "/search/v2/crawl",
    data=json.dumps({"url": "https://example.com/docs", "limit": 25, "maxDepth": 2, "excludePaths": ["/docs/archive/.*"]}).encode(),
    headers={"Authorization": "Bearer " + os.environ["GALON_API_KEY"],
             "Idempotency-Key": "docs-2026-11-14",
             "Content-Type": "application/json"},
    method="POST",
)
try:
    with urllib.request.urlopen(req) as res:
        print(res.status, json.load(res))
except urllib.error.HTTPError as err:  # the error body carries detail.code and detail.next_action
    print(err.code, json.load(err))

Node.js 18+ (fetch)

const res = await fetch(`${process.env.GALON_API_BASE}/search/v2/crawl`, {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.GALON_API_KEY}`, "Idempotency-Key": "docs-2026-11-14", "Content-Type": "application/json" },
  body: JSON.stringify({url: "https://example.com/docs", limit: 25, maxDepth: 2, excludePaths: ["/docs/archive/.*"]}),
});
console.log(res.status, JSON.stringify(await res.json()));

5. Read the crawl's pages. status is scraping, completed, failed or cancelled; 20 pages per answer, follow next while present. Results are kept 24 hours. DELETE /search/v2/crawl/{id} stops a running job.

curl

curl -sS -X GET "$GALON_API_BASE/search/v2/crawl/{id}" \
  -H "Authorization: Bearer $GALON_API_KEY" \
  -w '\n%{http_code}\n'

Python 3 (standard library)

import json, os, urllib.error, urllib.request

req = urllib.request.Request(
    os.environ["GALON_API_BASE"] + "/search/v2/crawl/{id}",
    headers={"Authorization": "Bearer " + os.environ["GALON_API_KEY"]},
    method="GET",
)
try:
    with urllib.request.urlopen(req) as res:
        print(res.status, json.load(res))
except urllib.error.HTTPError as err:  # the error body carries detail.code and detail.next_action
    print(err.code, json.load(err))

Node.js 18+ (fetch)

const res = await fetch(`${process.env.GALON_API_BASE}/search/v2/crawl/{id}`, {
  method: "GET",
  headers: { Authorization: `Bearer ${process.env.GALON_API_KEY}` },
});
console.log(res.status, JSON.stringify(await res.json()));

Concepts

Scrape, map, crawl. POST /v1/search/v2/scrape reads one page; /map lists a site's links; /crawl starts a job (GET /v1/search/v2/crawl/{id} pages through its results, DELETE cancels it). A SDK written for the market's v2 API migrates by changing its base URL and key; an option Galon does not serve is refused by name, never ignored.

Polite by design. One request per second per host, robots.txt obeyed (unreachable robots.txt: the site is not read), user agent GalonBot/1.0 (+https://galon.ai/bot), public pages only: private, local and reserved networks are refused.

Try it in the console. Services → Search → Read websites: one address, one button; the page's Markdown, the links, a site reading, and this first call with your project's address.

Counting. One page for scrape and crawl, one call for map; a refused page is not counted. Monthly pages per project and per account, daily requests per project: 429 by name when reached.

Retention. Crawl results are kept 24 hours; scrape and map keep no content.

API reference

These routes are served by the code but are not in the OpenAPI contract yet:

MethodPathAuthWhat it does
POST/v1/search/v2/scrapekey search:queryRead one page
POST/v1/search/v2/mapkey search:queryList a site's links
POST/v1/search/v2/crawlkey search:queryStart a crawl
GET/v1/search/v2/crawl/{id}key search:queryRead a crawl's results
GET/v1/search/v2/crawl/{id}/errorskey search:queryPages a crawl could not read
DELETE/v1/search/v2/crawl/{id}key search:queryCancel a crawl

Errors

Every error is JSON: {"detail": {"code", "message", "next_action"}}. next_action.code is stable, next_action.hint says what to do next, and console_url is added when a person must act. The codes below are those this service returns.

StatusCodeWhennext_action
403scope_missingThe key lacks search:query.add_missing_scope
400field_unknownA field the API does not accept.fix_request
400url_invalidNot an absolute http(s) URL on a standard port.fix_request
400url_blockedA private, local or reserved address (also after a redirect).fix_request
400robots_disallowedThe site's robots.txt forbids the page.fix_request
400format_unsupportedA format that is not served (screenshot, json, …).fix_request
400javascript_rendering_unavailablewaitFor or actions (JavaScript rendering).fix_request
400option_unsupportedAn option that is not served (headers, location, webhook, prompt…).fix_request
429rate_limitedToo many requests to one site; Retry-After says when.retry_after_delay
429key_rate_limitedToo many POST requests with one key in a minute (default 60); Retry-After.retry_after_delay
429daily_limit_reachedThe daily request limit is reached.raise_limit_or_plan
429quota_exceededThe monthly page quota is reached.raise_limit_or_plan
429crawl_limit_reachedTwo crawl jobs already run for this project.raise_limit_or_plan
409idempotency_conflictThe same Idempotency-Key with a different request.resolve_conflict
404crawl_not_foundNo crawl with this id.check_resource_id
404crawl_expiredThe crawl's results are older than 24 hours.check_resource_id
next_actionWhat to do
add_missing_scopeThis key does not have the scope this call needs. Create a key with that scope in the console, then retry the same call with the new key.
fix_requestThe request is invalid. Fix it as 'message' and 'field' say, using the OpenAPI contract, then send it again. Do not retry it unchanged.
retry_after_delayToo many requests. Wait the Retry-After delay (seconds), then retry with exponential backoff and the same Idempotency-Key.
raise_limit_or_planA plan quota or a spending limit is reached. Raise the limit or change the plan in the console, or wait for the next period, then retry.
resolve_conflictThe request conflicts with the resource state or with an earlier request under the same Idempotency-Key. Read the resource first; for a new intent, use a new Idempotency-Key.
check_resource_idNothing with this id is visible to this key. List the resources to get a valid id, and check that the key belongs to the right project and environment (test or live).

Limits, free tier and pricing

Read from the platform code each time this page is served.

LimitValue
Crawl pages (default limit)10
Crawl pages (maximum per job)100
Crawl depth (maximum)5
Map links (maximum)1,000
Page size (bytes)5 MiB
Results kept (hours)24
Concurrent crawl jobs per project2
Pages per site per day (default)500
Site reading requests per minute per key (default)60
Page timeout (milliseconds, maximum)60,000
Indexes per project (default)10
Documents per index (default)100,000
Documents per account (default)500,000
Document size64 KiB
Documents per batch1,000
Queries per second per key (default)20

Usage prices, in US dollars, read from the price grid served by the platform each time this page is served. Paid from the prepaid balance.

ItemPriceUnit
page0.741 USDper 1,000 documents
call0.741 USDper 1,000 requests

Billing units: one page for scrape and crawl, one call for map, paid from the prepaid balance.

FAQ

Is this a web search API?

No: it reads pages and sites you name. Search over an index is not on this page.

Can it render JavaScript?

No: the page is read as served. waitFor and actions are refused by name.

Can I try it without code?

Yes: in the console, Services → Search → Read websites.

How big can a crawl be?

100 pages and 5 levels per job, 15 minutes; the request's limit above 100 is capped and the answer says so (warning).

More