Recently, Google's dominant position in search engines has caused distress among publishers. According to a Bloomberg report, Google is leveraging its market influence to force publishers into a dilemma: either allow their content to appear in Google's AI responses or risk a decline in search rankings. This has left many publishers, who rely heavily on Google search traffic, feeling helpless.
Image source note: The image was generated by AI, provided by the image licensing service Midjourney
Google's crawler, Googlebot, is used for both search and AI summaries. Although website owners can use the "nosnippet tag" to prevent certain content from being used in AI responses, doing so may jeopardize their overall search visibility. Because completely blocking Google's crawler could result in losses that many publishers cannot bear. Simply blocking the AI crawler is not a solution, as the content for AI responses comes from the main crawler.
Many publishers are now in a difficult position. Joe Ragazzo, editor of Talking Points Memo, said: "This is like a survival crisis for these companies. Either opt out and face immediate death, or cooperate with Google, but ultimately it might just be a slow death, as Google may eventually no longer need them."
In response to Engadget, Google stated that preventing the entire article from being used in AI summaries does not prevent the crawler from seeing the "full text" used for ranking or appearing in web search results. Publishers can also choose to block only certain parts of their website from appearing in AI summaries.
According to insiders, Google has halted negotiations with publishers regarding potential licensing agreements for using their content in AI responses. In contrast, AI companies like OpenAI and Perplexity, along with Apple, are planning to compensate publishers through direct compensation or revenue sharing from ads. However, these companies do not have the resources to transform the entire internet into a chatbot ecosystem.
Google's $60 million agreement with Reddit is a good example, where Google tries to obtain the most AI content at the lowest cost. As a result, posts from Reddit sometimes surpass the original publishers' content in search results, which undoubtedly creates an unfair impact on the broader web. Moreover, Reddit, in order to protect its data, has even started blocking index crawlers from other search engines, creating a historic phenomenon: a website exclusively for a specific search engine.
Key points:
📌 Google uses its market advantage in search engines to force publishers into a choice: participate in AI responses or risk losing search exposure.
📌 Publishers can use the "nosnippet tag" to prevent content from being used in AI responses, but this may affect overall search rankings.
📌 Google has stopped negotiating content usage licenses with publishers, while some AI companies are trying to address the issue through compensation.