Commit 6422115c authored by Your New SJW Waifu's avatar Your New SJW Waifu 💬

Add robots.txt

parent 32358b8a
Pipeline #23191 passed with stages
in 9 minutes and 1 second
......@@ -8,6 +8,7 @@ npm run build
rsync -ra public/ "${TARGET}/instance/static"
cp dist/index.html "${TARGET}/instance/static/index.html"
cp robots.txt "${TARGET}/instance/static/robots.txt"
rsync --delete -ra dist/static/ "${TARGET}/instance/static/static"
rsync --delete -ra images/ "${TARGET}/instance/static/images"
rsync --delete -ra sounds/ "${TARGET}/instance/static/sounds"
......
# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
User-agent: *
Disallow: /web
Disallow: /settings
Disallow: /login
Disallow: /user-search
Disallow: /registration
Disallow: /cgi-bin
Disallow: /pages/DMCA.html
Disallow: /oauth
Disallow: /search
Disallow: /tag
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment