Free tool

    Robots.txt tester

    Paste your robots.txt and a page path to see which search and AI crawlers can reach it.

    • GooglebotAllowed
    • BingbotAllowed
    • GPTBotBlocked
    • PerplexityBotAllowed
    • ClaudeBotAllowed
    • Google-ExtendedAllowed
    • CCBotAllowed

    Let Rank Pilot fix it for you

    Free website analysis. Automated fixes from £29/mo.

    Analyse my website freeStart free