広告が嫌いですか? 行く広告なし今日 

Robots.txt テスター

開発者ネットワーキング

広告・取り除く？

入力

自動処理

出力

クライアント側

広告・取り除く？

ガイド

Robots.txt テスター

Paste your robots.txt file and test whether specific URLs are allowed or blocked for any crawler. Validates syntax against RFC 9309, highlights errors, and extracts sitemap references.

使い方

Paste your robots.txt content into the input field, enter a URL path to test, and select a User-agent. The tool instantly shows whether the URL is allowed or blocked, which rule matched, and any syntax issues in your robots.txt file.

特徴

RFC 9309 Compliant Parsing – Spec-compliant robots.txt parser with proper directive precedence
URL Pattern Matching – Full wildcard support with * and $ patterns
User-Agent Selection – Test against Googlebot, Bingbot, GPTBot, ClaudeBot, and more
Syntax Validation – Highlights malformed directives, unknown fields, and common mistakes
Matching Rule Display – Shows exactly which Allow/Disallow rule matched and why
Sitemap Extraction – Lists all Sitemap directives found in the file
Crawl-Delay Detection – Identifies crawl-delay settings per User-agent
Example Robots.txt – Pre-loaded examples for quick testing