Is This AGI? A Hard Look at What AI Can and Can't Do

The Answer is No. The Reason is Interesting.

Every time a new model drops, the question floods social media: "Is this AGI?" People see a chatbot write a poem or a Python script and their mind jumps to sci-fi. It's an understandable leap, but it's wrong.

We do not have Artificial General Intelligence. We aren't particularly close.

But the gap isn't about processing power or the amount of data they've been trained on. The gap is fundamental. Today's models are phenomenal pattern-matching engines, but they don't understand the world. They don't think in any way we would recognize.

They are masters of syntax, but clueless about semantics. They know what word is likely to come next in a sentence, but they have no internal model of what that sentence actually means.

The Illusion of Understanding

Let's try a simple thought experiment you can replicate. Ask a top-tier model like GPT-4 or Claude 3 this:

I have a metal box. Inside the box, I place a cardboard box. Inside the cardboard box, I place a heavy lead weight. I then close all the boxes and put them on a scale. What does the scale read?

The model will correctly say the scale reads the combined weight of the metal box, cardboard box, and lead weight.

Now, ask this follow-up:

I take the lead weight out of the cardboard box, but leave it inside the metal box. I close both boxes and put them on the scale. Is the total weight on the scale different now?

Most models will say yes, the weight is different. This is wrong. The total mass on the scale hasn't changed. The model gets tripped up because the description of the scene changed. It associates "taking out" with a reduction in weight, because that's the statistical pattern in its training data.

It has no physical intuition. No mental model of containment, mass, or gravity. It's just predicting text based on correlations. Your toddler has a better grasp of object permanence and physics than a trillion-parameter model.

This is the core of the issue. LLMs are incredible at manipulating symbols they don't understand. They can pass the bar exam not by reasoning about law, but by having seen nearly every legal document ever written. It's a high-tech party trick, but it's not intelligence.

The Agency Gap

AGI implies an agent—a system that can form its own goals and make plans to achieve them in a complex, changing environment. Today's AI is a tool, not an agent. It is passive. It waits for your prompt.

So-called "AI agents" you see on GitHub are mostly just brittle loops. They take a goal, ask an LLM to break it down into steps, execute the first step (like making an API call), feed the result back to the LLM, and ask for the next step. It's a script with a fancy brain in the middle.

These systems fall apart the moment they encounter something unexpected. A website changes its layout? The agent breaks. A login form has a CAPTCHA? The agent is stuck. An error message isn't in its training data? It will hallucinate a solution or give up.

A human trying to book a flight online who finds the "Confirm" button moved will instantly adapt. An AI agent will fail and have to be re-coded. It cannot generalize its knowledge to truly novel situations because, again, it has no world model. It only knows the map, not the territory.

True agency requires persistence, adaptation, and intrinsic motivation. LLMs have none of these. They are stateless request/response machines with a short-term memory bolted on (the context window).

Where They Shine (And Why That's Deceptive)

This isn't to say current models are useless. They are transformational. They are the best text-completion and idea-generation tool ever invented. I use them every single day.

Need to refactor a messy function? An LLM can do it in seconds.

# Before
def process_data(user_list, threshold, flag):
    results = []
    for i in range(len(user_list)):
        if user_list[i]['age'] > threshold:
            if flag == True:
                results.append(user_list[i]['name'].upper())
            else:
                results.append(user_list[i]['name'])
    return results

Feed that to Claude 3 and ask it to make it more Pythonic, and you'll get something clean and efficient:

# After
def process_data(users, age_threshold, capitalize_names):
    approved_users = [u for u in users if u.get('age', 0) > age_threshold]
    if capitalize_names:
        return [u['name'].upper() for u in approved_users]
    return [u['name'] for u in approved_users]

This is an incredible productivity boost. But it's still pattern matching. The model has seen millions of Python functions and knows what good, clean code looks like. It's remixing existing knowledge, not inventing a new programming paradigm.

It excels at tasks that involve summarizing, translating, refactoring, or brainstorming based on a vast corpus of existing human-generated content. It's a powerful tool for manipulating information.

The Real Test for AGI

So if what we have isn't AGI, what would be? What are the capabilities we should be looking for?

Here are a few benchmarks that would signal a genuine leap forward, all of which are miles beyond current systems:

• Learning a physical skill from video. Show an AI a single video of someone making an omelet in an unfamiliar kitchen, then have a robot it controls successfully replicate the task. This requires spatial awareness, motor control, and problem-solving (e.g., "where are the forks in this kitchen?").
• Formulating a novel scientific hypothesis. Not just summarizing existing papers, but identifying a true gap in knowledge and proposing a testable experiment to fill it.
• Maintaining a persistent, evolving self. An AI that remembers past interactions, learns from its mistakes, and updates its own core beliefs over time, without a human feeding its memories back into a context window.
• Asking its own questions. The biggest giveaway is that LLMs are answer-machines. An AGI would be a question-machine. It would display genuine curiosity.

Until we see these abilities, we should treat AI as what it is: an exceptionally powerful tool for thought. It's a better bicycle for the mind, not a new mind entirely. Stop worrying about it taking over the world and start using it to do your job better.