Skip to main content
POST
Robots.txt Generator

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json
ioComment
string

Header Comment (optional)

ioDefaultDisallow
string

Disallow paths

ioDefaultAllow
string

Allow paths (optional)

ioDefaultCrawlDelay
number

Crawl-delay (seconds, optional)

Required range: x >= 0
ioBotAgent
enum<string>

User-agent

Available options:
,
Googlebot,
Googlebot-Image,
Bingbot,
YandexBot,
DuckDuckBot,
Baiduspider,
GPTBot,
ChatGPT-User,
Google-Extended,
CCBot,
anthropic-ai,
ClaudeBot,
PerplexityBot,
Twitterbot,
AhrefsBot,
SemrushBot,
Applebot,
__custom__
ioBotAgentCustom
string

Custom user-agent

ioBotDisallow
string

Disallow paths

ioBotAllow
string

Allow paths (optional)

ioBotCrawlDelay
number

Crawl-delay (seconds, optional)

Required range: x >= 0
ioSitemapUrls
string

Sitemap URL(s)

Response

Tool output

tool
string
required
tool_version
string
required

Output-contract version for this tool. Currently "1" for all tools.

outputs
object
required
credits_used
integer
required

Credits this call consumed, after any settlement refund. 0 when metering is disabled.

request_id
string
credits_remaining
integer | null

Credits left in the current monthly allowance, or null when metering is disabled.