Reddit calls out Microsoft for using its content without paying

Reddit CEO Steve Huffman is calling out Microsoft and others to pay if they want to continue scraping the site’s data.

The move comes in the wake of Reddit striking a deal with Google and OpenAI.

"Without these agreements, we don’t have any say or knowledge of how our data is displayed and what it’s used for, which has put us in a position now of blocking folks who haven’t been willing to come to terms with how we’d like our data to be used or not used," Huffman said in an interview this week, according to The Verge.

Reddit CEO specifically called out Microsoft, Perplexity, and Anthropic for not negotiating and said that "it has been "a real pain in the ass to block these companies."

Reddit has been escalating the fight against crawlers. At the start of July, it updated its robot.txt file to block web crawlers whom with it did not have an agreement.

Later, the Reddit results were available only on Google, which pays Reddit for its data, and not on other search engines such as Microsoft's Bing.

Huffman said that Microsoft has been using Reddit's data to train its AI and in Bing results without informing them and Reddit data has been sold to other search engines via Bing API.

"We’ve had Microsoft, Anthropic, and Perplexity act as though all of the content on the internet is free for them to use," Huffman said. "That’s their real position."

Microsoft’s head of search, Jordi Ribas, said on X: "Reddit has blocked Bing from crawling their site for search, favoring another search engine and impacting competition from Bing and Bing-powered engines."

Anthropic spokesperson Jennifer Martinez told The Verge, "Reddit has been on our block list for web crawling since mid-May and we haven’t added any URLs from Reddit to our crawler since then. We respect robots.txt, the industry-accepted signal for blocking web crawling."

Image Source: Unsplash