# Please do not crawl faster than 1 hit per second. Thanks. # User-agent: * Disallow: /cgi-bin/ Disallow: /Cellar/jamesn/ User-agent: grub-client Disallow: /cgi-bin/ Disallow: /Cellar/ Disallow: /UPCDB/ User-agent: ZyBorg Disallow: /cgi-bin/ Disallow: /Cellar/ Disallow: /UPCDB/