Armory
Source
Browse
MCPs

sharozdawa-robotstxt-ai

Analyze and generate robots.txt files with AI crawler detection for 20+ bots including GPTBot and ClaudeBot.

Score
22.5571 signal
Evidence
1 star
Last commit
as last read from GitHub; most reads are from 2 Sep 2026 or later
Listed

Install

No one-command install. Set it up from its source.

Alternatives · MCPs

  1. just-every-read-website-fast161 stars · 26 forks93.326
  2. nushey-agents-md-generator6 stars · 1 fork55.168
  3. mcp-fetch-node6 stars · 1 fork55.168

What it is

MCP server RobotsTxt AI by sharozdawa, catalogued on PulseMCP. Analyze and generate robots.txt files with AI crawler detection for 20+ bots including GPTBot and ClaudeBot.

When to use it

Analyze and generate robots.txt files with AI crawler detection for 20+ bots including GPTBot and ClaudeBot.

Notes

Listed from the PulseMCP registry. The registry does not state a license. Check it before production use.