— SEO & GEO

Robots.txt Tester

Paste your robots.txt, enter a path and a crawler, and instantly see whether it is allowed or blocked — and exactly which rule decided it.

ALLOWEDVerdict
googlebotGroup
Matched Allow: / (matched 1 chars).
— What it does

The Robots.txt Tester is a free tool that checks whether a given URL path is allowed or blocked for a specific crawler by your robots.txt file. It selects the matching User-agent group with a * fallback, applies the longest-match-wins rule between Allow and Disallow, and supports the * wildcard and the $ end-anchor — the same logic Google uses. It runs entirely in your browser, so nothing is uploaded.

  • Reports ALLOWED or BLOCKED and shows the exact rule that matched.
  • Implements Google’s group selection, longest-match-wins, * wildcard and $ end-anchor.
  • Test any path against any user-agent, defaulting to Googlebot.
  • Runs entirely in your browser — your robots.txt is never uploaded to a server.

How to use it

1

Paste your robots.txt

Paste the full contents of your robots.txt file into the input box — all groups and rules.

2

Enter a path and crawler

Type the URL path to test, such as /blog/post, and the user-agent to test as, defaulting to Googlebot.

3

Read the verdict

See ALLOWED or BLOCKED instantly, along with the matching Allow or Disallow rule and the group it came from.

Frequently asked

How does robots.txt decide between Allow and Disallow?
When both an Allow and a Disallow rule match a URL, the rule with the longest path pattern wins. If they are the same length, Allow takes precedence. This tester applies that exact longest-match-wins logic.
What do the * and $ symbols mean?
A * in a path matches any sequence of characters, so Disallow: /*.pdf blocks every PDF. A $ anchors the match to the end of the URL, so Disallow: /*.php$ blocks URLs ending in .php but not /file.php?id=1.
How is the right user-agent group chosen?
Crawlers use the most specific matching User-agent group. This tester looks for a group whose User-agent matches the one you entered, and falls back to the User-agent: * group if there is no specific match.
Does this tool upload my robots.txt anywhere?
No. The matching runs entirely in your browser with JavaScript. Your robots.txt content and test paths are never sent to a server, so the tool is private and works offline.

Related tools

Robots.txt Generator

Build a clean, valid robots.txt — allow or block crawlers, set a user-agent, add crawl-delay and your sitemap, then copy or download.

Open tool →

XML Sitemap Generator

Paste a list of URLs and get a valid sitemap.xml — with changefreq, priority and lastmod — ready to copy or download.

Open tool →

Canonical Tag Generator

Paste a URL and get a clean, copy-ready rel=canonical link tag to tell search engines which version of a page is the original.

Open tool →

← All free tools

— Built by saavos

These tools are free. So is the first version of your agent.

saavos is the AI agent that lives on your website — themed to match your design, answering visitors from your own content, and telling you what they actually want to know. Paste your URL and see it answer, before you install anything.

Make my site feel alive