Disallow robots from indexing /interact/ (#10666) · c4f2433300 - mastodon - code.as

Pārlūkot izejas kodu

Disallow robots from indexing /interact/ (#10666)

This does not provide any new information and may just triple the number
of crawled pages

master^2

ThibG pirms 5 gadiem

committed by

Eugen Rochko

vecāks

0db269f3dc

revīzija

c4f2433300

1 mainītis faili ar 1 papildinājumiem un 0 dzēšanām

Dalītais skats

							
								+ 1
								
								- 0
							
public/robots.txtParādīt failu
								
			@@ -2,3 +2,4 @@
		
			User-agent: *
		
			Disallow: /media_proxy/
		
			Disallow: /interact/