Mydentify AI Crawler Access Checker 1.1.4
A dependency-free Node.js checker and browser evaluator for inspecting how a public page treats documented OpenAI and Anthropic crawler names. It reads robots.txt, checks page-level directives and X-Robots-Tag headers, and sends labeled user-agent requests. Results describe observable handling only; probes originate from the caller and do not prove provider-IP behavior, indexing, training, or authorship.
| Tags | ai-crawlers robots-txt seo developer-tools security artificial-intelligence security software-development |
|---|---|
| License | MITL |
| State | stable |