# AllTheWeb User-agent: fast-webcrawler Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # AltaVista User-agent: scooter Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # Googlebot User-agent: googlebot Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # Looksmart User-agent: wisenutbot Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp User-agent: zyborg Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # MSN User-agent: msnbot Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # Yahoo Vertical Crawler User-agent: yahoo-verticalcrawler Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # Internet Archive User-agent: ia_archiver Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp #Ask Jeeves User-agent: askjeeves Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp #DIGGIT User-agent: digger Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp #Infoseek User-agent: infoseek Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp #Lycos User-agent: lycos Crawl-Delay: 10 Disallow: */imagens Disallow: /backup Disallow: /cgi_bin Disallow: /files Disallow: /music Disallow: /photos Disallow: /novo Disallow: /site Disallow: /wp # Exclude every other bot (hopefully) User-agent: * Disallow: /