![]() ![]() If you block it your shared post look w/o any info, only a url.Ģ.- Crawl-delay as far as I know is not a standar so some bots don´t care about it. Facebook bot (or Twitter bot) don´t crawl your site only fecht your site to extract info for your shared post (Image, publisher, date….). Saying this, if you share a post in FB then FB bot MUST see your site to extract the info it need for the shared post. ^.*(Adsbot|Ahrefs|MJ12bot|Seznam|Baiduspider|Yandex|SemrushBot|DotBot|spbot|Facebook).*$ I understand this question doesn’t really relate to theme issues, but wondered if you knowledgeable peeps could help?ġ.- Facebook is not excluded with the code you provided, need to add Facebook to the chain: ![]() Should I just remove the code when no ads are running and if I did this would this resolve the issue or are there other bots that might visit uninvited? Are there any security or privacy issues with the bot?Ĥ. – Setup a crawl delay in robots.txt to inform bots that read it to rate limit requests. ^.*(Adsbot|Ahrefs|MJ12bot|Seznam|Baiduspider|Yandex|SemrushBot|DotBot|spbot).*$ He has provided some code as follows that he think might help: He provided a snippet of data which showed that it visited my site 25 times in one day even though I don’t have any ads currently running. The contact has come back and thinks it could be related to a facebook bot (I have inserted code on my site). I thought it was related to my images so I changed them all but the issue persists apparently. I’ve been contacted by my hosting service provider regarding my site’s ‘high resource usage’. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |