Cynosphere/nitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

Block search engines via robots.txt (#631)

Browse source 
Prevents instances from being rate limited due to being senselessly
crawled by search engines. Since there is no reason to index Nitter
instances, simply block all robots. Notably, this does *not* affect link
previews (e.g. in various chat software).
This commit is contained in:
minus 2022-06-04 15:48:25 +00:00 committed by GitHub
parent 778c6c64cb
commit c543a1df8c
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
public/robots.txt Normal file
View file

@ -0,0 +1,2 @@
User-agent: *
Disallow: /