Robots.txt Tester
Robots.txt Tester Paste your robots.txt file and test whether specific URLs are allowed or blocked for any crawler. Validates syntax against RFC 9309, highlights errors, and extracts sitemap references. How to Use Paste your robots.txt content into the input field, enter a URL path to test, and select a User-agent. The tool instantly shows whether ...
Try the Tool »