mikros/rust
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Rollup merge of #59213 - kornelski:robots, r=Mark-Simulacrum

Browse Source 
Track changes to robots.txt

Currently `robots.txt` of doc.rust-lang.org is not part of any repo, so there's [no way to contribute any changes to it](https://internals.rust-lang.org/t/deadlock-about-fixing-outdated-documentation-links-in-search-engines/9374), such as needed for #44894 and countless dupes of this issue.

I propose adding it to this repo. I'm not in control of the infrastructure, so I can't help to automate deployment of it, but even just having the file under source control is IMHO a step forward.
This commit is contained in:
kennytm 2019-03-24 15:32:42 +08:00 committed by GitHub
commit 6c5e1d151a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 19 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
src/doc/robots.txt Normal file
View File

@ -0,0 +1,19 @@
# NB: This file is not automatically deployed. After changes, it needs to be uploaded manually to doc.rust-lang.org
User-agent: *
Disallow: /0.3/
Disallow: /0.4/
Disallow: /0.5/
Disallow: /0.6/
Disallow: /0.7/
Disallow: /0.8/
Disallow: /0.9/
Disallow: /0.10/
Disallow: /0.11.0/
Disallow: /0.12.0/
Disallow: /1.0.0-alpha/
Disallow: /1.0.0-alpha.2/
Disallow: /1.0.0-beta/
Disallow: /1.0.0-beta.2/
Disallow: /1.0.0-beta.3/
Disallow: /1.0.0-beta.4/
Disallow: /1.0.0-beta.5/