pdf scrapers
[disclosr.git] / robots.txt
Alexander Sadleir 1 # www.robotstxt.org/
2 # www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
3
Maxious 4 User-agent: *
Maxious 5 Disallow: /admin/
6 Sitemap: http://orgs.disclosurelo.gs/sitemap.xml.php