Crawl Sitemap
Retrieve crawl Sitemap with the Context.dev API. Supply a domain. Sample fields include keywords and languages. Use the result as input for a related analysis or application. The GET request schema also lists includeSubdomains, maxLinks.
- Selected price
- $0.002 / billing creditPublished pricing
- Sellers / implementations
- 1 / 1
- Requests · 30d
- —Loading...
- Success rate · 30d
- —Loading...
| Seller | Pricing | Requests · 30d | Performance | Access |
|---|---|---|---|---|
context-devCurrent Context.dev — Crawl Sitemap · GET | $0.002 / billing credit | — | —successP50— |
Run sends a real API request. Charges follow the published API contract.
Response structure
domainstringkey_metadataobjectcredits_consumedinteger
request_idstringsuccessbooleanurlsobject[]descriptionstringkeywordsstring[]languagestringtitlestringurlstring
// Inferred from response examples. Fields are optional and other fields may be returned.
export type ApiResponse = {
domain?: string;
key_metadata?: {
credits_consumed?: number;
};
request_id?: string;
success?: boolean;
urls?: Array<{
description?: string;
keywords?: Array<string>;
language?: string;
title?: string;
url?: string;
}>;
};- Method
- GET
- Endpoint
https://api.openany.si/v1/agent-apis/context-dev/v1/web/scrape/sitemap- Content type
- application/json
- Authentication
Authorization: Bearer $OPENANY_API_KEY
curl \
'https://api.openany.si/v1/agent-apis/context-dev/v1/web/scrape/sitemap?domain=context.dev&maxLinks=1' \
-X GET \
-H "Authorization: Bearer $OPENANY_API_KEY" \
-H 'Content-Type: application/json'Input parameters
Fields in this listing; additional options and input encoding can vary by provider.
| Parameter | Location | Type | Required | Description |
|---|---|---|---|---|
domain | URL query parametersquery | string | Yes | Domain to map, e.g. `stripe.com`. minLength: 3 |
includeSubdomains | URL query parametersquery | boolean | No | Include URLs on subdomains. |
maxLinks | URL query parametersquery | integer | No | Maximum number of URLs to return. minimum: 1 · maximum: 100000 |
sitemapUrl | URL query parametersquery | string | No | Fetch this sitemap instead of discovering sitemaps. Must belong to the domain or a subdomain. |
urlRegex | URL query parametersquery | string | No | Regex pattern. Only URLs matching this pattern are returned and counted against maxLinks. maxLength: 256 |
excludeUrlRegex | URL query parametersquery | string | No | Regex pattern. URLs matching this pattern are not returned, even when they match urlRegex. maxLength: 256 |
search | URL query parametersquery | string | No | Filter URLs by a topic or phrase, most relevant first. minLength: 2 · maxLength: 200 |
zdr | URL query parametersquery | string | No | `enabled` turns on zero data retention. Returns 403 `ZDR_NOT_ENABLED` unless your organization has ZDR. enum: "enabled", "disabled" |
Required markers inside a nested object or variant apply when that object or variant is used.
Sample response
Sample response · Documented HTTP 200 Published documentation example. Verify the actual response after an approved call.
{
"domain": "context.dev",
"key_metadata": {
"credits_consumed": 1
},
"request_id": "63cde093-1036-44f3-97d2-798a471f911f",
"success": true,
"urls": [
{
"description": "Context.dev is the web data API for AI agents: scrape any URL into LLM-ready Markdown, search the live web, crawl entire sites, and extract structured data with one REST API.",
"keywords": [
"web data api",
"web data api for ai agents",
"scrape api",
"crawl website",
"web crawler",
"web search api",
"markdown scraper",
"extract structured data",
"ai data extraction",
"web scraping for ai agents",
"llm web scraping",
"rag scraping api",
"brand api"
],
"language": "en",
"title": "Context.dev: Web Data API for AI Agents (Scrape, Search, Crawl)",
"url": "https://www.context.dev"
}
]
}Usage billing
A billing credit measures provider usage. Credits consumed depend on the operation and request options; one credit is not necessarily one call.
Agent integration
Copy the resource contract without sharing an API key.
View API Markdown
# Crawl Sitemap
Retrieve crawl Sitemap with the Context.dev API. Supply a domain. Sample fields include keywords and languages. Use the result as input for a related analysis or application. The GET request schema also lists includeSubdomains, maxLinks.
## Available implementations
context-dev · Context.dev — Crawl Sitemap: https://openany.si/markets/apis/capabilities/context-dev-crawl-sitemap?offering=d8c8599b-8ec2-40b9-9a4a-9c67e25db443 · $0.002 / billing credit
## Selected API contract
Documentation: https://openany.si/markets/apis/capabilities/context-dev-crawl-sitemap
Method: GET
Endpoint: https://api.openany.si/v1/agent-apis/context-dev/v1/web/scrape/sitemap
Price: $0.002 / billing credit
Pricing note: A billing credit measures provider usage. Credits consumed depend on the operation and request options; one credit is not necessarily one call.
Provider: context-dev
Contract revision: 3
Price source: OpenAny public API contract and documented provider unit rates. Price update time: not supplied.
Use OPENANY_API_KEY from the environment. API Key calls use your shared balance.
## Prepared request
```bash
curl \
'https://api.openany.si/v1/agent-apis/context-dev/v1/web/scrape/sitemap?domain=context.dev&maxLinks=1' \
-X GET \
-H "Authorization: Bearer $OPENANY_API_KEY" \
-H 'Content-Type: application/json'
```
## API contract
```json
{
"resource_type": "api",
"name": "Context.dev — Crawl Sitemap",
"endpoint": "https://api.openany.si/v1/agent-apis/context-dev/v1/web/scrape/sitemap",
"method": "GET",
"content_type": "none",
"execution_mode": "sync",
"authentication": {
"type": "bearer",
"environment_variable": "OPENANY_API_KEY"
},
"inputs": [
{
"description": "Domain to map, e.g. `stripe.com`.",
"location": "query",
"min_length": 3,
"name": "domain",
"nullable": false,
"required": true,
"type": "string"
},
{
"description": "Include URLs on subdomains.",
"location": "query",
"name": "includeSubdomains",
"nullable": false,
"required": false,
"type": "boolean"
},
{
"description": "Maximum number of URLs to return.",
"location": "query",
"maximum": "100000",
"minimum": "1",
"name": "maxLinks",
"nullable": false,
"required": false,
"type": "integer"
},
{
"description": "Fetch this sitemap instead of discovering sitemaps. Must belong to the domain or a subdomain.",
"location": "query",
"name": "sitemapUrl",
"nullable": false,
"required": false,
"type": "string"
},
{
"description": "Regex pattern. Only URLs matching this pattern are returned and counted against maxLinks.",
"location": "query",
"max_length": 256,
"name": "urlRegex",
"nullable": false,
"required": false,
"type": "string"
},
{
"description": "Regex pattern. URLs matching this pattern are not returned, even when they match urlRegex.",
"location": "query",
"max_length": 256,
"name": "excludeUrlRegex",
"nullable": false,
"required": false,
"type": "string"
},
{
"description": "Filter URLs by a topic or phrase, most relevant first.",
"location": "query",
"max_length": 200,
"min_length": 2,
"name": "search",
"nullable": false,
"required": false,
"type": "string"
},
{
"description": "`enabled` turns on zero data retention. Returns 403 `ZDR_NOT_ENABLED` unless your organization has ZDR.",
"enum_values": [
"enabled",
"disabled"
],
"location": "query",
"name": "zdr",
"nullable": false,
"required": false,
"type": "string"
}
],
"billing": {
"currency": "USD",
"mode": "usage",
"priceMicroUsd": "2000",
"rounding": "ceil",
"unit": "credit",
"unitQuantity": "1"
},
"examples": [
{
"http_status": 200,
"name": "Documentation example (source abbreviated)",
"request": {
"body": null,
"headers": {},
"path": {},
"query": {
"domain": "context.dev",
"maxLinks": 1
}
},
"response_body": {
"domain": "context.dev",
"key_metadata": {
"credits_consumed": 1
},
"request_id": "63cde093-1036-44f3-97d2-798a471f911f",
"success": true,
"urls": [
{
"description": "Context.dev is the web data API for AI agents: scrape any URL into LLM-ready Markdown, search the live web, crawl entire sites, and extract structured data with one REST API.",
"keywords": [
"web data api",
"web data api for ai agents",
"scrape api",
"crawl website",
"web crawler",
"web search api",
"markdown scraper",
"extract structured data",
"ai data extraction",
"web scraping for ai agents",
"llm web scraping",
"rag scraping api",
"brand api"
],
"language": "en",
"title": "Context.dev: Web Data API for AI Agents (Scrape, Search, Crawl)",
"url": "https://www.context.dev"
}
]
}
}
]
}
```Error handling
| Status | Next step |
|---|---|
400 | Check the offering and request fields. |
401 / 403 | Verify your credentials and access permissions. |
402 | Inspect the selected access mode and returned payment requirements. |
429 | Respect the returned rate limit and retry guidance. |
5xx | Inspect the error and usage record before retrying a paid operation. |
Documentation example (source abbreviated)
Published documentation example. Verify the actual response after an approved call.
Prepare your environment
Use a terminal with cURL installed.
Use an OpenAny API key with shared-balance access. Confirm availability and price before running; a call may charge your balance.
Set the key in your terminal using the hidden prompt below. Keep its value out of source files and logs.
Terminal · macOS / Linuxprintf 'OpenAny API key: ' read -r -s OPENANY_API_KEY export OPENANY_API_KEY printf '\n'Run the request
Paste the complete command below into your terminal.
Values to replace before running 8
Review these request fields before running. Keep the published provider and request method unchanged.
domain- Domain to map, e.g. `stripe.com`.
includeSubdomains- Include URLs on subdomains.
maxLinks- Maximum number of URLs to return.
sitemapUrl- Fetch this sitemap instead of discovering sitemaps. Must belong to the domain or a subdomain.
urlRegex- Regex pattern. Only URLs matching this pattern are returned and counted against maxLinks.
excludeUrlRegex- Regex pattern. URLs matching this pattern are not returned, even when they match urlRegex.
search- Filter URLs by a topic or phrase, most relevant first.
zdr- `enabled` turns on zero data retention. Returns 403 `ZDR_NOT_ENABLED` unless your organization has ZDR.
Terminal · cURLcurl \ 'https://api.openany.si/v1/agent-apis/context-dev/v1/web/scrape/sitemap?domain=context.dev&maxLinks=1' \ -X GET \ -H "Authorization: Bearer $OPENANY_API_KEY" \ -H 'Content-Type: application/json'Check the response
Sample response · Documented HTTP 200
Sample response{ "domain": "context.dev", "key_metadata": { "credits_consumed": 1 }, "request_id": "63cde093-1036-44f3-97d2-798a471f911f", "success": true, "urls": [ { "description": "Context.dev is the web data API for AI agents: scrape any URL into LLM-ready Markdown, search the live web, crawl entire sites, and extract structured data with one REST API.", "keywords": [ "web data api", "web data api for ai agents", "scrape api", "crawl website", "web crawler", "web search api", "markdown scraper", "extract structured data", "ai data extraction", "web scraping for ai agents", "llm web scraping", "rag scraping api", "brand api" ], "language": "en", "title": "Context.dev: Web Data API for AI Agents (Scrape, Search, Crawl)", "url": "https://www.context.dev" } ] }
Common questions
Do I need a separate key for every provider?
Use your OpenAny API key for standard API access. Read it from OPENANY_API_KEY in your environment and keep it out of prompts, source code and logs.
Are these providers interchangeable?
No. Each offering has its own inputs, pricing and response contract. Selecting a catalog offering does not establish automatic routing or fallback. Confirm the service contract before calling.
How is this API billed?
API-key calls use your shared balance at the selected implementation's published rate. Check the billing unit and request-specific pricing rules before making a paid call. Recorded usage determines the final charge under that contract.
Where do the price and contract come from?
The selected implementation publishes its provider, request contract, revision and billing rules through OpenAny's public API catalog. A revision number identifies a contract version, not a price update date. A price update timestamp is not supplied; do not treat a page date as one.
How do OpenAny Balance and x402 differ?
OpenAny Balance uses an API key and one shared balance. Card, USDC and x402 top-ups, card auto top-up and USDC subscriptions fund that balance; available routes appear on the Credits page. Direct x402 is a separate pay-per-request flow, available only on listings that show it and requiring an authorized payment client.
Does Playground send a paid request?
Yes. After you sign in, Run sends a real API request using an active OpenAny API key. Charges follow the published API contract and use your shared balance. Selecting an implementation, editing inputs and viewing sample responses do not send requests.
Use Crawl Sitemap with your agent
Inspect endpoint contracts and confirm availability and pricing before a paid call.