spaCy/website/robots.txt.jade

mirror of https://github.com/explosion/spaCy.git
//- 💫 ROBOTS.TXT

if environment != "deploy"
    | User-agent: *
    | Disallow: /

if ALPHA
    | User-agent: Googlebot
    | Disallow: /
Update to new website 2016-10-31 18:04:15 +00:00			`//- 💫 ROBOTS.TXT`
Update website 2016-10-03 18:19:13 +00:00
Fix robots and meta 2017-06-05 18:07:52 +00:00			`if environment != "deploy"`
Update website 2016-10-03 18:19:13 +00:00			`\| User-agent: *`
			`\| Disallow: /`
Fix robots and meta 2017-06-05 18:07:52 +00:00
			`if ALPHA`
			`\| User-agent: Googlebot`
			`\| Disallow: /`