August 14, 2026  |  SIGN UP

Jim Cooper
DIGIDAY, EDITOR-IN-CHIEF

European publishers are getting crushed by AI bots. As we report this morning, research from TollBit found that compared to their North American counterparts, European sites get scraped more often, receive fewer referrals and have their robots.txt rules more aggressively flaunted.

It seems the multitude of languages spoken in Europe make it easier for scraper work arounds. Or perhaps it’s the diversity of data employed? Whatever the cause, site porousness is causing referral damage over there. Scrapes on European sites were nearly 20% higher in June than in January 2026. Local news and sports sites saw the highest quarter-over-quarter rise. And the scrape-to-referral ratio went from 150:1 in Q1 2026 to 227:1 in Q2 2026, according to the Tollbit report, which features a lot of scary stats.

One more for you: The median European site’s robots.txt instructions to not scrape was ignored nearly three times more often than North American sites.

ADVERTISEMENT

THE BUSINESS OF AI 

European publishers are getting hit harder by AI bot scraping, report finds

Median AI scrapes per site were four-times higher on European sites compared to North American sites. And European publishers receive just one human referral visit from AI apps for every 179 AI bot visits — a rate more than three times worse than for North American sites.

ZERO-CLICK FUTURE

CMOs are struggling to link AI visibility with sales

AI visibility isn’t the only zero-click concern keeping marketers up at night. Now that tools like Scrunch and Profound are commonplace, CMOs are thinking ahead to the next task: finding evidence that a strong zero-click search presence actually drives sales.

PARTNER INSIGHTS FROM MOLOCO

The problem with last-touch for CTV

Measuring CTV on last-touch alone tells only part of the story. Across hundreds of advertisers, 30% of campaigns are significantly undervalued by last-touch models alone.

MARKETING ON PLATFORMS

Lifestyle brand Stanley 1913 adapts its marketing for the AI search era

The brand could chase quick wins the way some competitors have, gaming its way into AI recommendations. Instead, it’s playing the long game, treating this less as a sprint than a marathon it needs to actually train for.

A MESSAGE FROM DIGIDAY

Publishers: How are you rethinking audience strategies in the AI era?

AI is transforming how audiences discover and consume content, prompting publishers to rethink engagement, subscriptions and revenue strategies. In this new survey, Digiday and Braze want to know how publishers are investing in stronger first-party relationships, using data to better understand audiences and preparing for the future of publishing.

Take the survey

AI REVENUE GENERATION

Perplexity blocks Time’s ads served to AI agents, calling them ‘deceptive’

Less than two weeks after Digiday reported that Time had started serving ads inside the markdown versions of its webpages, aimed at AI agents rather than human readers, Perplexity confirmed that it blocked those ads from influencing its own search index.

MEMBER EXCLUSIVE

Media Briefing: The case for and against markdown for AI bots

Not everyone is convinced markdown pages necessary. AI bots can already interpret standard webpages, skeptics argue. The debate also highlights a broader question facing publishers: is better visibility in AI worth making their content even easier for bots to scrape?

PARTNER INSIGHTS FROM TALON.ONE

Why marketers must become AI agent-ready now

Agentic AI is projected to account for 15%–25% of total U.S. e-commerce by 2030, according to Bain & Company — an indication that businesses won’t truly be omnichannel unless they have incorporated agentic shopping experiences.