# Robots.txt file for http://stavanger.katolsk.no/ # Disallow Direct Hit Grabber User-Agent: Direct Hit Grabber User-Agent: Mozilla/4.06 [en] (Direct Hit Grabber) Disallow: / # Disallow Inktomi's Web robot User-Agent: Slurp Disallow: / # Disallow Scooter User-Agent: Scooter/2.0 Disallow: / # Let all other crawlers do their stuff unhindered User-Agent: * # Begrensninger User-Agent: * Disallow: /graf User-Agent: * Disallow: /filsmaa