User-agent: * Disallow: /wp-admin/ Disallow: /tag/ Disallow: /*? Disallow: /*comment-page User-agent: MSIECrawler Disallow: / User-agent: psbot Disallow: / User-agent: Fasterfox Disallow: / User-agent: Xenu Disallow: / User-agent: SiteSucker Disallow: /