MEMO
robos.txt에서 차단해야할 봇

User-agent:AhrefsBot
User-agent:Amazonbot
User-agent:Arachni
User-agent:Baiduspider
User-agent:baiduspider
User-agent:Baiduspider+ 
User-agent:bbot
User-agent:BLEXBot
User-agent:brands-bot
User-agent:dataforseo-bot
User-agent:DotBot
User-agent:Exabot
User-agent:Eyeotabot
User-agent:MegaIndex
User-agent:MJ12bot
User-agent:PetalBot
User-agent:SemrushBot
User-agent:WordPress
Disallow: / 

이 게시물을

공유하기

SEARCH

MENU NAVIGATION