#Tell crawlers not to crawl some folders. This save bandwith and server resource. User-agent: * Crawl-delay: 10 # Directories Disallow: /images/ Disallow: /admin/ Disallow: /img/ Disallow: /local/ Disallow: /dimmingdiv/ Disallow: /includes/ Disallow: /mobile/