mitmproxy/docs/bucketassets/robots.txt

mirror of https://github.com/mitmproxy/mitmproxy.git
User-agent: *
Disallow: /archive/
Disallow: /master/
Disallow: /dev/
docs: add robots.txt and error.html - robots.txt prevents crawling of everything but /stable - error.html has a simple meta refresh to the latest stable docs. This will also help crawlers find the right path. 2018-03-04 21:57:50 +00:00			`User-agent: *`
			`Disallow: /archive/`
master branch -> main branch 2021-03-16 15:17:27 +00:00			`Disallow: /master/`
			`Disallow: /dev/`