Warn crawlers not to touch the tripwire
OpenDev’s static content sites implement web application firewall
rules designed to catch and block malicious crawlers by instructing
them not to index a particular non-existent file and then blocking
any client that tries to reach it. Add a corresponding robots.txt
directive to the docs.openstack.org site similar to the one used in
https://docs.opendev.org/robots.txt already.
Change-Id: I73c5e8c027e559ad731636560277463c3f87d7bd
Signed-off-by: Jeremy Stanley fungi@yuggoth.org