User-agent: Mediapartners-Google Disallow: Crawl-delay: 20 User-agent: * Disallow: /*.aspx Disallow: /*.ashx Disallow: /v2/ Disallow: /r/ Disallow: /login/esia/ User-agent: * Disallow: /cgi-bin Crawl-delay: 20 User-agent: BLEXBot Crawl-delay: 30 User-agent: BLEXBot Disallow: /