Bots Overwhelm Wikipedia, Driving Creators to New Platforms

The Challenge of AI Bot Traffic on Wikipedia

Artificial intelligence (AI) chatbots are causing a significant increase in traffic to Wikipedia, raising concerns about the impact on human users. This surge has prompted the Wikimedia Foundation, the organization behind Wikipedia, to create a separate platform specifically for bots. The goal is to ensure that the online encyclopedia remains accessible and functional for regular users.

In April, the foundation reported a dramatic rise in bot traffic, which has put a strain on the site’s bandwidth and increased maintenance costs. If this pressure continues, it could hinder access for people and affect the foundation’s ability to raise donations, which are crucial for sustaining the service.

Selena Deckelman, the chief product and technology officer at the Wikimedia Foundation, explained during an interview at Wikimania, an annual gathering for Wikipedia contributors held in Nairobi, that a new site for bots was developed to reduce their impact on the main platform. “We’ve definitely seen an increase in bot traffic, and because of that, it can negatively affect humans’ ability to get to the website,” she said.

Deckelman emphasized that the foundation has been working on this issue for several years, not just in response to AI but as a long-term solution. She encouraged companies using bots to utilize the enterprise API, a tool designed to manage traffic more efficiently. “What we’re asking that folks that want to scrape our site, if they move that traffic over to the enterprise API so that we can prioritize human traffic over bot traffic.”

The Role of the Enterprise API

The enterprise application programming interface (API), launched in 2021, was initially intended for high-volume users such as search engines and voice assistants. Unlike Wikipedia’s main site, the enterprise API is a paid service, serving as an additional revenue stream for the foundation.

However, its recent focus on handling bot traffic comes in response to a sudden spike in activity that could potentially overwhelm the site. Deckelman noted that during major events, such as the announcement of a new Pope, there was a significant increase in traffic. “What we saw is that one of the network pipes filled up, because there was so much activity and a lot of that is coming from bots,” she said.

Despite these efforts, the enterprise API has not yet become a strong source of revenue. About 34 percent of the foundation’s funding still comes from donations triggered by the “donate” banner on Wikipedia. This share could decrease if human traffic continues to decline. In the year ending June 2024, donor numbers increased by about 0.5 million, but the average donation amount dropped from $11.38 to $10.58.

The Impact of Chatbots on Wikipedia Traffic

As AI chatbots become more popular, fewer people are visiting Wikipedia directly. Instead, they are accessing its content through AI tools, which means they are less likely to see the “donate” banners. According to DataReportal figures, ChatGPT surpassed Wikipedia in monthly page visits this year, with a 43 percent increase from 3.53 billion to 5.06 billion by July. Meanwhile, Wikipedia’s traffic decreased from 3.93 billion to 3.83 billion during the same period and is expected to continue declining.

This shift in user behavior poses a challenge for the Wikimedia Foundation. It must find ways to maintain accessibility for human users while also adapting to the growing presence of AI tools. By creating a dedicated platform for bots, the foundation hopes to balance these competing demands and ensure that Wikipedia remains a reliable and sustainable resource for everyone.