Does Robots.txt Apply to Subdomains and HTTP vs HTTPS?
Why every subdomain, protocol and port needs its own robots.txt and how to keep them consistent.
How to Test Robots.txt in Google Search Console
How to confirm what Google sees in your robots.txt using the Search Console report and URL Inspection.
Robots.txt Returns 404 or 500: What Google Does and How to Fix It
How Google treats a missing or erroring robots.txt and how to fix both situations.
How to Block Images From Google With Robots.txt
How to use the Googlebot-Image user agent to keep images out of Google Images, and what to avoid.
Robots.txt User-Agent Groups: Which Rules Does Googlebot Follow?
How crawlers pick a user-agent group in robots.txt and why specific groups override the wildcard group.
Do Meta Keywords Still Matter for SEO?
Why the meta keywords tag no longer matters for Google and what to do on your own site.






