Reddit results not showing up in many search engines, except Google: Report

Many search engines are either not showing the full site result or showing old content from Reddit, which recently entered into a deal with Google for AI models being trained on its data

Don't want to miss the best from Business Standard?

Reddit's recent actions in restricting access to its data for search engines other than Google have stirred significant attention and debate within the tech community. Initially, Reddit forged a prominent partnership with Google earlier this year, aimed at integrating Reddit's vast dataset into Google's AI models. This collaboration resulted in Reddit's content gaining heightened visibility in Google Search results, surpassing even the original websites linked within Reddit posts.

However, this exclusivity appears to have extended beyond mere collaboration. Reddit has recently updated its Robots Exclusion Protocol (robots.txt) to prohibit all bots from indexing any part of its site. This move, as stated by Reddit, aims to curb the misuse of public content, particularly by commercial entities engaged in scraping Reddit's data for various purposes without consent.

The rationale behind Reddit's stricter control over its content includes concerns over unauthorised scraping and potential misuse of its data. Although not officially confirmed, the adjustment to their robots.txt file coincides with their increased involvement in AI training, suggesting a possible motivation to protect their data integrity and value.

This shift has effectively blocked other search engines from crawling and indexing Reddit's content comprehensively. Reports indicate that non-Google search engines either display outdated information or fail to show Reddit results altogether, consolidating Google's dominance in showcasing Reddit content.

Critics argue that Reddit's actions reflect a broader strategy to safeguard its data assets, attract investors, and diversify revenue streams. The platform's purported warnings to Google about leveraging its data for AI training without compensation underscore Reddit's evolving stance on data ownership and commercial use.

Reddit's decision to limit search engine access may underscores its strategic pivot towards data protection and monetisation, influencing how its vast community-generated content is accessed and utilised across digital platforms.