week6-- Can Wikipedia Survive the Age of Crawlers and AI? --WU WENHAO

WU WENHAO,2021080464

Summary

As artificial intelligence rapidly evolves, the way we create and consume knowledge is also undergoing a fundamental shift. The Wikimedia Foundation’s recent article, How crawlers impact the operations of the Wikimedia projects, reveals how Wikipedia—one of the internet’s most important open knowledge sources—is facing growing strain from automated web crawlers. These bots, especially those used to train large language models (LLMs), are massively scraping Wikipedia and Wikimedia Commons content without proper attribution or restraint.

This traffic is not just a technical nuisance—it’s altering how knowledge is accessed and valued. While Wikimedia’s infrastructure was designed to serve human readers and volunteer editors, it now finds itself overwhelmed by non-human activity that accounts for 65% of its most expensive traffic. Most of this comes from bots downloading huge quantities of multimedia files, such as images and videos, which increases operational costs and crowds out resources meant for real users. If left unchecked, this could jeopardize the future stability of Wikipedia’s open-access model.

Interesting Discovery

What I found especially surprising is how much invisible labor and cost goes into keeping Wikipedia freely available—not just for readers, but for bots too. I always thought of Wikipedia as a public resource mainly used by people like me, but this article made me realize that a large chunk of traffic now comes from machines, not humans. It’s kind of ironic: Wikipedia was built by humans, for humans, but now it's being quietly consumed by AI systems that don’t even acknowledge it.

I think the example about Jimmy Carter’s page is a good way to show how much things have changed. When he passed away, the page got a huge spike in views, which you’d expect—but what really caused problems was the constant background traffic from bots downloading videos and images. That’s wild to me. It shows that even though we may not see it, the open knowledge we rely on is under a lot of pressure behind the scenes.

This made me think about how many of the AI tools we use every day are probably trained on Wikipedia data. But I never stopped to wonder whether they gave anything back. It’s like everyone is benefiting from it—except Wikipedia itself.

Discussion Questions

Is it fair for automated programs to use human-created content without contributing back? I think this raises a big question about what ‘open access’ really means in the age of AI.

Comments

Post a Comment

Popular posts from this blog

Introduction to the blog

This blog is a digital forum where students from Collective Intelligence in Practice: Understanding Wikipedia at the Hanyang University in Korea (2025) will discuss their weekly readings and activities. Each week you are expected to post a blog post on our blog and make a comment on another student's post. You can do so on one of the following three topics: a) You can blog about the readings for the next week. As you are expected to do the assigned readings FOR the lectures they are assigned to, you are expected to post your blog and comments BEFORE that lecture. An ideal commentary should be about half a page in length (~200 words), divided into three clearly labeled sections in which you: 1) summarize in your own words of materials you read; 2) mention any new, interesting, or unusual items learned; 3) identify at least one question, concern, or discussion angle that is either problematic in some respect or could have been elaborated more. This question is likely to be incor...
Read more

Week4 - Review about the readings for the next week. - Jo HyeonSeong (조현성)

 1. Summarize in my own words of materials. The book is primarily about the openness of Wikipedia. Openness means that anyone can participate in the creation and editing of articles without any qualifications. Openness can be a double-edged sword. While it means that you can edit without having expert knowledge of a topic, it also means that you can participate even if you have expert knowledge. For experts, articles written by non-experts can come across as inaccurate knowledge, which can anger experts. This anger can also be a catalyst for Wikipedia to continue to improve. However, articles edited only by non-experts can cause misunderstandings and convey incorrect information, so openness can be considered a double-edged sword. In fact, it sometimes causes some conflict, such as Wales' negative stance. Openness also means freedom from copyright, which means anyone can take it and use it anywhere. That is why the knowledge of non-experts can have greater impact. That is why appro...
Read more

Week 1. My Recent Wikipedia Edits - Jeong seolah (정설아)

I made several edits to the Wikipedia page for 'Daegu'.  Daegu, located in the southeastern part of South Korea, is the third-largest urban area in the country, following Seoul and Busan. With a population exceeding 2.3 million, it is the fourth-largest metropolitan city in South Korea and the second-largest in the Yeongnam region after Busan. Often referred to together with the surrounding North Gyeongsang Province as Daegu-Gyeongbuk, this region is home to over 5 million residents. Known for its rich history and vibrant economy, Daegu stands as a key player in South Korea's cultural and industrial landscape. Below is a summary of the changes I contributed. 1) Summarize and link Sights Section:  - Improved clarity by changing “mountains keep many traditional temples” to “the mountains keep many traditional temples.” Mosque Controversy Section:  - Simplified a sentence by removing “and of the authorities.” Festivals Section:  - Made phrasing smoother by changing “tho...
Read more