# robots.txt for https://www.tegeldwerg.be/ # All robots will spider the domain User-agent: * Disallow: /admin/ Disallow: /cron/ Disallow: /inc/ Disallow: /smartoptimizer/ User-agent: Googlebot Disallow: /admin/ Disallow: /cron/ Disallow: /inc/ Disallow: /smartoptimizer/ Sitemap: https://www.tegeldwerg.be/sitemap.xml