Insult to injury? AI bots come for your content and site resources

Generative AI is changing content as we speak. The trouble is that none of us know what is going to happen. 

And now there is yet another spanner in the proverbial works. 

Impact on SEO

As it stands, content publishers like us invest a great deal in creating useful content. Useful content brings in organic traffic and high-intent visitors to our sites. 

Traffic is critical to revenue, as every content marketer knows. However, it is in decline. 

More people are turning to AI tools for the insights we provide. Zero effort on their part, and without so much as a nod in our direction. 

Granted, there are plenty of caveats about accuracy right now. But these tools will only get better. Anyone who disagrees is sticking their head in the sand. 

Some publishers have taken steps to prevent AI bots from scraping their content. Many have tweaked their robots.txt file prohibiting scrapers.

Bombarded by bots

The only problem is that we’ve seen evidence that certain AI bots aren’t respecting rules. 

In fact, their repeated requests are almost as bad as a DDoS attack. Bots are depleting server resources, driving up hosting bills, and impacting site performance.

ClaudeBot has been a particularly egregious offender. It has bombarded sites with excessive crawling in the last two weeks. 

Here are some stats from our firewall: 

Thankfully, we have MalCare’s firewall mitigating this onslaught for our sites. Because of its smart bot protection, we know that the good bots will have access, whereas the others stay out.

Intellectual property and all that jazz

So, as it stands, an AI tool can scrape all our insights for free. AND we get to pay astronomical bills for the privilege. All while losing traffic that is critical to us. 

Hardly seems fair.

But the truth is we’re not sure how to feel about it all. 

Take our WordPress security plugin comparison, for example. We spent months testing 13 top security plugins, throwing malware and attacks at them. We sought an objective answer to the question: will this security plugin protect a WordPress site? 

On the one hand, it is annoying to have months of hard work repurposed so easily, where we don’t appear to reap any of the benefits. 

On the other hand, AI tools that scrape useful, well-researched content like ours end up being a new channel for discovery. 

Do we want to deny MalCare’s top billing in this new channel? 

We’ll have to wait and see.