Technical
Robots.txt
A plain-text file that asks crawlers not to fetch certain paths. Controls fetching, not indexing.
Where to go next
Related terms
Definitions here are written and maintained by about-seo.com, owned and run by Trusted Media. Nothing on this page is generated per visitor.