User-Agent: * Disallow: /jobs/ User-Agent: ia_archiver Disallow: / User-Agent: * Allow: /