User-Agent: * Disallow: /stat/ Allow: / User-agent: * Crawl-delay: 120 User-agent: Googlebot Crawl-delay: 120 User-agent: googlebot-image Crawl-delay: 120 User-agent: Twiceler www.cuill.com/robots.html Crawl-delay: 120 User-agent: Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html) Crawl-delay: 120 User-agent: Twiceler Crawl-delay: 120 User-agent: Twiceler www.cuill.com/twiceler/robot.html Crawl-delay: 120 User-agent: Twiceler-0.9 http://www.cuill.com/twiceler/robot.html Crawl-delay: 120 User-agent: Mail.Ru/1.0 Crawl-delay: 120 User-agent: Mail.Ru Crawl-delay: 120 User-agent: Slurp Crawl-delay: 120 User-agent: yahoo-slurp Crawl-delay: 120 User-agent: ia_archiver Crawl-delay: 120 User-agent: Gaisbot/3.0 Crawl-delay: 120 User-agent: Gaisbot Crawl-delay: 120 User-agent: baiduspider Crawl-delay: 120 User-agent: panscient.com Crawl-delay: 120 User-agent: Naverbot Crawl-delay: 120 User-agent: Naver Crawl-delay: 120 User-agent: Gigabot Crawl-delay: 120 User-agent: Yeti Crawl-delay: 120 User-agent: Charlotte Crawl-delay: 120 User-agent: Speedy Crawl-delay: 120 User-agent: Yandex Crawl-delay: 120 User-agent: Sogou web spider Crawl-delay: 120 User-agent: Sogou spider Crawl-delay: 120 User-agent: * Crawl-delay: 120 User-agent: msnbot Crawl-delay: 10 User-agent: Yeti User-Agent: Baiduspider User-Agent: Baiduspider+ User-Agent: BaiduImagespider User-agent: CazoodleBot User-agent: dotbot User-agent: panscient User-agent: panscient.com User-agent: Wget User-agent: Sogou Disallow: /