503 Responses on robots.txt Blind Googlebot to Page-Level 503 Headers
During emergency server maintenance, serving HTTP 503 (Service Unavailable) on robots.txt halts all Googlebot crawling across the domain because the crawler cannot verify crawl permissions before accessing HTML pages.