Is AI capable of new thought?
Day 136 / 366
Everyone is chirping about how great OpenAI’s new model is and how we are all going to lose our jobs or die or something. They see a few superficial examples and get excited, and don’t dig any deeper. Most of the time they don’t bother to try those things themselves, just believe what they saw in a pre-recorded demo video by the people who made the model and who will profit from its popularity.
If we are willing to praise AI by looking at a few great examples, then I think we should give equal attention to the times when it does horribly wrong as well.
For instance, there is a theory that LLMs don’t come up with any new answers. They just present you with things they have already seen in their training data. After all, they have been trained from the data scraped from the internet. It’s not that surprising if it answers a commonly known puzzle. It does not think of the answer from scratch, it just constructs it from similar examples it has seen already.
Again, this is just a theory, but here is an example that supports it. You must have heard of the popular puzzle where a farmer has a bag of seeds, a fox, and a chicken. And he needs to cross the river on a boat. The catch is that the boat can only hold 2 things at a time. You can search for the solution online, but the gist of it is that it involves him taking two items at a time, leaving one, and then coming back with one. Now if I ask ChatGPT for the answer, it will surely come up with it.
But what if I change the question to make it extremely simple? If I say that there is just a man and a goat, and they have a boat. There isn’t even any limit on the capacity of the boat. Will ChatGPT give me the correct answer?
This is what the leading AI model right now answered. If it was capable of coming up with a solution from scratch, it would have just said that the man and the goat should just get on the boat and cross the river. Or maybe it could have asked me some questions for clarification. But it did not. It looked at the question, related it to the chicken, fox, and beans questions it was trained on, and just tried to fit an answer in the same template.
Do we seriously think this AI is going to take our jobs?
What is crazier is that when people point out these flaws, they get bombarded with comments trying to defend the LLM! They would say things like “You are using it wrong!”, “A human would have gotten that wrong as well on the first try”, or “This is just the beginning! Wait and see what GPT-5 will be like!”. It is truly bizarre.
Don’t get me wrong. LLMs are amazing. But we should not get carried away and start saying that they are intelligent, or capable of reasoning. It is a Large Language Model, trained on a large amount of text data. And it’s sole purpose is to try and come up with the next word (or token). It just happens that in doing so it can write code, or translate languages, or answer basic factual questions. But that is because all those things were part of its training data. And we cannot rely on them to do these things correctly most of the time.