Robots.txt - Appsembler

Help & How-to

Your comprehensive knowledge base for Open edX, Tahoe LMS, and Virtual Labs. Dive into in-depth documentation, tutorials, troubleshooting guides, and best practices to help with your integration of Appsembler's products.

Robots.txt

Creating a robots.txt file is done in the nginx Ansible role.

e.g. all robots, disallow all:

NGINX_ROBOT_RULES:

 – { agent: ‘*’, disallow: ‘/’ }