Sitemap URL Discovery (by Domain)

omnial/web-scrape-sitemap

Given a domain (or an explicit sitemap URL), returns the list of URLs found in that site's sitemap, along with how many sitemap files were discovered, fetched, and skipped, and whether any sitemap-level errors occurred. When sitemaps are discovered but none are fetched, `urls` comes back empty even though `success` is true.

activeper callv6

Price

$0.00346 per call

You pay the price shown. If a call turns out to cost more than its quote, you pay the actual price, capped at twice the quote. No monthly minimum. Top-ups start at $0.50.

Data source

The requested web page

Data from the web page you request.

Use it

Five steps: search, inspect, learn, run, settle. No per-tool setup; your agent reaches this tool through the meta-tools it already has.

Connect once and your AI can find and run this tool, and any other it needs.

Install it Create an account

agent

// 1. search: find the tool
omnial_search({ query: "what you need" })

// 2. inspect: price and input schema
omnial_inspect({ tool: "omnial/web-scrape-sitemap" })

// 3. learn: read the usage guide at runtime
omnial_learn({ tool: "omnial/web-scrape-sitemap" })

// 4. run: price the exact call first, then run it
omnial_execute({ tool: "omnial/web-scrape-sitemap", input: { /* see input_schema */ }, dry_run: true })
omnial_execute({ tool: "omnial/web-scrape-sitemap", input: { /* ... */ } })

// 5. settle: the call is charged when it finishes, nothing to do

Not connected yet? Point your MCP client at https://mcp.eden-engine.com/mcp, then copy a config for your client.

Pricing

Every charge this tool can incur.

Charge	Price
Per call	$0.00346

Updated: Oct 25, 2026

Input and output schemas

Input schema

input_schema.json

{
  "type": "object",
  "required": [
    "domain"
  ],
  "properties": {
    "zdr": {
      "enum": [
        "enabled",
        "disabled"
      ],
      "type": "string",
      "description": "Set 'enabled' to bypass the upstream service's shared caches and omit request and response content from its retained usage logs. Requires zero data retention on the upstream organization, otherwise the call fails with ZDR_NOT_ENABLED."
    },
    "domain": {
      "type": "string",
      "minLength": 3,
      "description": "Domain to build a sitemap for, e.g. 'example.com'."
    },
    "maxLinks": {
      "type": "integer",
      "default": 10000,
      "maximum": 100000,
      "minimum": 1,
      "description": "Maximum number of links to return. Default 10000, max 100000."
    },
    "urlRegex": {
      "type": "string",
      "maxLength": 256,
      "description": "RE2-compatible pattern. Only matching URLs are returned and counted against maxLinks."
    },
    "timeoutMS": {
      "type": "integer",
      "maximum": 300000,
      "minimum": 1000,
      "description": "Upstream timeout in milliseconds (max 300000). The upstream service aborts the call with a 408 when exceeded. Keep it below the endpoint's requestTimeoutMs so the provider answers before the platform's own budget expires."
    },
    "sitemapUrl": {
      "type": "string",
      "format": "uri",
      "description": "Explicit sitemap URL. When provided, exactly this sitemap is crawled instead of discovering the domain's sitemaps."
    }
  },
  "additionalProperties": false
}

Output schema

output_schema.json

{
  "type": "object",
  "required": [
    "domain",
    "success",
    "urls",
    "meta"
  ],
  "properties": {
    "meta": {
      "type": "object",
      "required": [
        "sitemapsDiscovered",
        "sitemapsFetched",
        "sitemapsSkipped",
        "errors"
      ],
      "properties": {
        "errors": {
          "type": "integer",
          "description": "Count of sitemap-level fetch/parse errors. Typically 0; a non-zero value's effect on `success` is unconfirmed."
        },
        "sitemapsFetched": {
          "type": "integer",
          "description": "How many of the discovered sitemaps were actually downloaded and parsed."
        },
        "sitemapsSkipped": {
          "type": "integer",
          "description": "How many discovered sitemaps were not fetched. Observed non-zero (2 of 2) on a real domain whose sitemaps were located but not retrieved within the call -- a caller cannot assume every discovered sitemap gets fetched."
        },
        "sitemapsDiscovered": {
          "type": "integer",
          "description": "How many sitemap files were located for the domain."
        }
      },
      "additionalProperties": true
    },
    "urls": {
      "type": "array",
      "items": {
        "type": "string"
      },
      "description": "The URLs found in the domain's sitemap. Observed populated (10 entries, capped by maxLinks) when a sitemap was discovered and fetched, and observed empty when sitemaps were discovered but none were fetched (see meta.sitemapsFetched) -- an empty array does not necessarily mean the domain has no sitemap."
    },
    "domain": {
      "type": "string",
      "description": "The domain the request targeted, echoed back."
    },
    "success": {
      "type": "boolean",
      "description": "Whether the run completed without a fatal error. Typically true, including when zero URLs are discovered -- a caller cannot infer from `success` alone whether any sitemap content was actually retrieved (see meta.sitemapsSkipped)."
    }
  },
  "description": "Sitemap discovery result for the requested domain, narrowed from the provider's whole result object: the URLs found in the site's sitemap, plus counts of how many sitemaps were found, fetched, and skipped, and whether any sitemap-level errors occurred.",
  "additionalProperties": true
}