robots-compliance-scrape-workflow-mcp-server
An MCP server for AI-agent web scraping that enforces robots.txt compliance and requires user confirmation of scrape plans, packaging scraped data with provenance evidence.
- Score
- 22.5571 signal
- Evidence
- 1 star
- Last commit
- as last read from GitHub; most reads are from 2 Sep 2026 or later
- Listed
Install
No one-command install. Set it up from its source.
Alternatives · MCPs
- firecrawl-mcp7,515 stars · 895 forks · passed install test99.852
- just-every-read-website-fast161 stars · 26 forks93.326
- ritvij14-crawl4ai24 stars · 7 forks79.537
What it is
An MCP server for AI-agent web scraping that enforces robots.txt compliance and requires user confirmation of scrape plans, packaging scraped data with provenance evidence.
When to use it
An MCP server for AI-agent web scraping that enforces robots.txt compliance and requires user confirmation of scrape plans, packaging scraped data with provenance evidence.
How to install / invoke
See Glama for the install config.
Notes
Listed from the Glama MCP registry.