# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file User-agent: * Disallow: / User-agent: Googlebot Disallow: User-agent: Facebot Disallow: # Infrawatch sends malformed requests and pollutes our error tracking. User-agent: Infrawatch Disallow: /