Robots.txtThis is a text file residing in your website directory that informs search engine robots how to visit your website. It tells search engines which web pages and directories they can and shouldn’t view.