Draft crawl rules and sitemap hints—copy to your site root
Treat robots.txt as a hint for compliant bots and pair it with auth, rate limits, and Search Console settings.