Ok but they do respect it, we know that https://searchengineland.com/microsoft-confirms-reddit-blocked-bing-search-444385
They even have a page telling you how to use it https://blogs.bing.com/webmaster/June-2008/Robots-Exclusion-Protocol-joining-together-to-pro/
Yeah but it stops bing and a bunch of AI scrapers that want to act like they’re following the rules
I don’t think they actually block malicious bots, the change they’ve made is just to the robots.txt, they don’t have to do anything.
Nah I’m pretty sure it’s still lemmy. This page seems to say they are looking at moving to something else, but they’re still using lemmy for now https://docs.beehaw.org/docs/important-questions-decisions-and-reflections/beehaw-lemmy-and-a-vision-of-the-fediverse/