Glossary

Robots.txt

A plain text file in a website’s root directory providing instructions to web crawlers.