# Robots.txt file from http://www.hgcms.net # All robots will spider the domain User-agent: * Disallow: /403.html Disallow: /404.html Disallow: /#Data#Backup#/ Disallow: /#Data#Base#/ Disallow: /Admin/ Disallow: /Ads/ Disallow: /Cert/ Disallow: /Editor/ Disallow: /Images/ Disallow: /Include/ Disallow: /NewCode/ Disallow: /Skin/ Disallow: /Style/ Disallow: /Upload/ Disallow: /User/ Allow: * /sitemap.xml /sitemap.txt