User-agent: * Allow: / Disallow: /admin/ # Try to not overflow dmca robot with amazon products (DOES NOT SEEM TO WORK THOUGH!) User-agent: DMCA.com Page Protection Crawling Service Disallow: /products/