Robots.txt
Checks status, redirects, content type, sitemap declarations, explicit AI crawler groups, broad blocks, and contradictory allow/disallow rules.
Sample report
The first portfolio run checked 5 public sites and found 0 issues, 15 warnings, and repeated missing `llms.txt` and explicit AI crawler policy surfaces.
Checks status, redirects, content type, sitemap declarations, explicit AI crawler groups, broad blocks, and contradictory allow/disallow rules.
Checks `/llms.txt`, `/.well-known/llms.txt`, and `/llms-full.txt` for basic Markdown structure and links.
Checks title, description, canonical, JSON-LD, noindex/nofollow, server-rendered text, and visible important links.