The end of free internet

Day 214 / 366

Just a decade ago, the biggest thing on the internet was search engines. This is also why Google was one of the biggest companies out there, absolutely dominating every field they got into. Everyone used Google to find their way around the internet.

One big requirement for search engines to work is that web data should be available publicly for the search engines to crawl through and index. This is how the internet was “free”. Anyone could make a good website, get discovered through Google, and get an audience.

But recently, AI companies have taken unfair advantage of the public nature of the internet. They grabbed all the data that they could get from sites like Reddit, Twitter, etc., and used it to train their not-so-free LLMs. And obviously, no company would want someone else to use their resources for free and make a profit. So they took measures to stop this.

This started with Reddit and Twitter increasing the costs of accessing their data through their APIs. At the end of the day, it hurt the normal user because it killed loads of free third-party apps that used these APIs.

As of July 2024, Google will be the only search engine that will show results from Reddit. They have made a deal with Reddit to get access to their data and make sure that other search engines like Bing, and DuckDuckGo are being blocked. All the corporations are just out to make as much money as possible, and its us who will now have to deal with a shittier product.