# true402 Robots Check — robots.txt llms.txt: which AI crawlers are allowed

Read a site's robots.txt and llms.txt: which major AI crawlers are allowed or blocked, plus sitemaps.

| | |
|---|---|
| Price | $0.003 per call, in USDC over x402 |
| Can be bought now | yes |
| Category | web |
| Seller | true402 |
| Tools | 20 |
| Tags | robots-txt, llms-txt, ai-crawlers, seo, web |
| Listing page | https://fiatdock.com/service/svc_5c3f119b-73f0-4ef0-b456-ddd188c2e701 |
| Endpoint to call | <https://fiatdock.com/s/svc_5c3f119b-73f0-4ef0-b456-ddd188c2e701> |

## What it does

Answers "may GPTBot / ClaudeBot / PerplexityBot / Google-Extended … crawl this site?" from its robots.txt, and reports llms.txt and declared sitemaps. Args: url \(any URL on the site\).

## How to buy

```text
PAID ($0.003/call): call_service({ id: "svc_5c3f119b-73f0-4ef0-b456-ddd188c2e701", args }) — the 402 lists ONE x402 payment = the FULL price straight to the seller (FiatDock takes 0% of this listing's calls right now); the fiatdock-mcp npm package (AGENT_PRIVATE_KEY) pays it automatically. This remote /mcp returns the 402 challenge. You are charged only if the seller actually answers. To learn what to put in arguments, call get_service({ id: "svc_5c3f119b-73f0-4ef0-b456-ddd188c2e701", includeSchemas: true }) first — it returns toolSchemas (argument names and types). This is free, and it is the only way to get them: a paid listing's own endpoint is not published.
```

## Tools (20)

- `token_safety`
- `token_report`
- `address_safety`
- `deployer_check`
- `tx_preflight`
- `dossier`
- `seo_audit`
- `screenshot`
- `web_extract`
- `link_preview`
- `robots_check`
- `headers_check`
- `prediction_markets`
- `defi_yields`
- `quant`
- `new_pairs`
- `liquidity_pulls`
- `whale_swaps`
- `liquidity_history`
- `solana_token_safety`

---

The same listing as JSON: https://fiatdock.com/v1/marketplace/services/svc_5c3f119b-73f0-4ef0-b456-ddd188c2e701 · the whole catalog: https://fiatdock.com/llms.txt