In the last 6 months, the number of websites blocking GPT bot went from 1.6% to 2.2% I applaud all companies that care about the ownership of their content and fair compensation for creators! Unfortunately, half of those blocking ChatGPT forgot to block Common Crawl, Anthropic/Claude, and other bots used for training LLMs. #LLM #ChatGPT
Jens Altmann
2w
Bringt ja nichts, wenn es Bots ignorieren oder Microsoft sagt, dass alles free to use ist https://finance.yahoo.com/news/exclusive-multiple-ai-companies-bypassing-143742513.html?guccounter=1
Jens H.
2w
Completly lost mindset. These are the last lifelines of the old SEOs who are still trying to save their business. This is such old-fashioned and desperate behavior, the topic will be over in two to three years (maybe even sooner) anyway. So whether people will still be typing a question into google to get some stupid and non-personalized 0815 answers or stupid SEO texts and ugly websites full of bullshit. As if Chat GPT and Co needed these stupid websites.
Are you trying to imply something with this? Personally, I believe blocking any bot should really be reserved for websites that have server capacity issues. If you think that blocking a GPT bot is going to prevent your content from being stolen then I really don't know what they say. We've all been copying each other and spinning content and ideas since the earliest days of advertising and marketing.
Too bad bots will just ignore that.
Nigel Turtle
2w
Nice perspective Malte Landwehr. That still looks like a tiny percentage. It seems like there is still a major lack of awareness out there despite the lawsuits and the social media commentaries. Perhaps also a lack of technical knowledge and capability ?
Very insightful, thanks for sharing Malte! What is the source of these figures?
Ojash Yadav
2w
Great insight! But I wonder for how long we’ll be able to keep up with this. New LLMs are rolling our faster than Google updates.
If bots continue to scrape content and resue it, content sites will get less traffic. Less traffic means less money from ads or subscriptions. Which means content sites will go bus and the bots will have less content to scrape. With less content to scrape, AI might resort to generating its own content. And create a vicious cycle – AI generated content becomes the source to be scraped, feeding false or biased information back into the system. It will be like "scrape-ception"....... .......a never-ending loop of "scraped content," where made-up content fuels more made-up content. Actually it sounds a bit like the way social media is already. Influential making stuff up and people quoting them.
Do it smart
2wIf the competition blocks AI scraping, your product will be used and thus it is free marketing. Had it happen. Will happen. Especially if your product is quite specific. Carelessly blocking AIs from scraping seems to be the worst AIO (AI-SEO?) advise ever.