PrivateBin/robots.txt

mirror of https://github.com/PrivateBin/PrivateBin.git synced 2024-03-22 13:10:41 +08:00
# Might as well keep robots away for performance and privacy reasons.
User-agent: *
Allow: /index.php
Disallow: /
For server-load and privacy reasons, disallow robots. 2012-08-02 23:23:04 +08:00			`# Might as well keep robots away for performance and privacy reasons.`
			`User-agent: *`
Allow discovery To allow search engines to at least find the website. 2013-01-30 03:33:31 +08:00			`Allow: /index.php`
Incorrect structure The structure for robots.txt is incorrect for some/most search engines. 2013-11-01 08:22:16 +08:00			`Disallow: /`