spaCy/website/robots.txt.jade

6 lines
95 B
Plaintext
Raw Normal View History

2016-10-31 18:04:15 +00:00
//- 💫 ROBOTS.TXT
2016-10-03 18:19:13 +00:00
if environment != "deploy" || ALPHA
2016-10-03 18:19:13 +00:00
| User-agent: *
| Disallow: /