User-agent: * Disallow: /~.bak Disallow: /~.cgi Disallow: /~.exe Disallow: /~.gif Disallow: /~.ico Disallow: /~.inc Disallow: /~.jpg Disallow: /~.log Disallow: /~.pl Disallow: /~.wav Disallow: /administrator/ Disallow: /cgi-bin/ Disallow: /classes/ Disallow: /components/ Disallow: /documentation/ Disallow: /editor/ Disallow: /images/ Disallow: /includes/ Disallow: /language/ Disallow: /mambots/ Disallow: /media/ Disallow: /modules/ Disallow: /templates/ Disallow: /uploadfiles/ Disallow: /usage/ Disallow: /.htaccess Disallow: /.htpasswd Disallow: /error404.html # Tells Scanning Robots Where They Are And Are Not Welcome # User-agent: can also specify by name "*" is for everyone # Disallow: if this matches first part of requested path, forget it # Tells Scanning Robots Where They Are And Are Not Welcome # # Worldwide TeleNet # Robot Exclusion File -- robots.txt # Author: Worldwide TeleNet # Last Updated: 01/25/04