Artificial intelligence is changing everything around us. But the question that always comes up is this — how do we know what tasks AI will be great at? Andrej Karpathy, one of the most respected voices in AI, offers a clear answer. It’s not about how creative or complex a task is. It’s about whether the task is verifiable.
He explains that tasks AI can verify and practice are the ones it improves at rapidly. This is where the future of automation lies.
In this post, I’ll explain exactly what that means and break everything down so you can get the full picture. Let’s break it down. But first, we’ll look at how he compares AI to a new type of software that learns, rather than one that follows fixed and hardcoded steps.
AI Is More Than Just Smart Software — It’s Software 2.0
Andrej Karpathy calls modern AI Software 2.0. It’s not a fancy name; it’s a whole new way of building software.
In the old model, or what he calls Software 1.0, people wrote exact instructions. Every task had to be broken into small steps. For example, if you wanted a calculator app, you had to write all the logic — what happens when someone presses + or =. Every rule had to be created by hand.
But Software 2.0 changes that completely. You don’t write rules. Instead, you show the system examples. Want AI to recognize cats in photos? You give it thousands of images labeled “cat” and “not cat.” The AI slowly adjusts itself by trial and feedback, until it gets good at spotting them on its own.
That means the “code” in Software 2.0 is mostly made up of data, training objectives, and feedback signals. It’s like teaching a child by showing examples, letting them try, and giving a gold star when they get it right.
This is why AI can now do things we couldn’t program before. We never wrote rules for language translation or self-driving cars. We just let the systems practice with data, again and again.
Software 1.0 vs. Software 2.0
To make this super clear, here’s a quick comparison:
| Feature | Software 1.0 | Software 2.0 |
|---|---|---|
| Written by humans | Yes | No |
| Learns from data | Rarely | Yes |
| Improves with practice | No | Yes |
| Example tasks | Typing, bookkeeping | Image recognition, code generation |
| Flexible logic | Limited | Very flexible |
In Software 2.0, AI writes itself by learning from examples. We guide it by showing it what “success” looks like.
That’s where verifiability comes in.
So, What Does Verifiability Actually Mean?
Now, here’s the key part of Karpathy’s insight. He says, the new question we should ask is — can we verify this task easily?
Verifiability means the AI can try a task, and there’s a clear way to know how well it did. It’s like having a scoreboard. If the AI makes a move, we can tell if that move was good or not.
Karpathy outlines three clear traits that make something verifiable:
– Resettable: The task can be started over and over again from the beginning.
– Efficient: The task can be repeated quickly, so the AI can do many attempts.
– Rewardable: There’s a clear way to score the result.
If these three are present, the task becomes a playground for AI learning.
Let’s use a simple example. Suppose you’re teaching AI to solve a maze. You can reset the maze after each try. The AI can go through it again and again. And when it reaches the end, you can give it a score based on how fast it got there. This is a highly verifiable task.
Now, imagine a job like managing a team of people or leading a business strategy meeting. These don’t reset. You can’t repeat the same exact meeting over and over. And there’s no clear score that says, “this was a 9/10 decision.” These tasks are less verifiable.
So, the more verifiable a task is, the easier it becomes for AI to get better at it through practice.
How Can We Check if a Task Is Verifiable
To figure out whether AI can learn and perform a certain task really well, there’s one big question to ask:
Can this task be practiced, measured, and improved over time without needing a human to explain what went right or wrong each time?
If the answer is yes, then that task is verifiable.
Let’s break that down with three key checks:
1. Can the task be repeated over and over?
This means the AI should be able to start from scratch each time and try again. Think of a puzzle game. You can hit “restart” and play the same level again and again.
This repeatability gives the AI more chances to practice and learn. If a task only happens once or is too complex to repeat in the same way, it becomes harder for AI to improve on it.
2. Can the AI do many attempts quickly?
Speed matters. The more times the AI can try something, the faster it learns.
If the task takes only a few seconds to run each time — like checking a sentence for spelling errors, the AI can practice millions of times and get very good at it.
But if each attempt takes hours or needs lots of setup, it becomes harder to scale. So fast, simple tasks are much easier to automate.
3. Can the results be checked automatically without human help?
This is about feedback. After trying something, the AI should be able to get a clear score or see a correct answer right away.
Let’s say the AI is translating a sentence into another language. If there’s a known good translation, it can compare its version and see how close it is. That gives it a way to measure how well it did.
Or imagine an AI solving math problems. The answer is either right or wrong. The computer can check this instantly, without a person saying “good job.”
If there’s no clear answer or the outcome depends on human opinion (like reviewing a painting), then the task becomes much harder for AI to learn on its own.
Final Thoughts on Andrej Karpathy’s Verifiability Factor
When you look at your own work or business, ask yourself: Is this task verifiable?
If the answer is yes, there’s a strong chance AI can learn it. When a task is easy to test and repeat, the technology keeps improving at it. This understanding can guide smarter decisions — whether you’re choosing a career, creating a product, or shaping a process.
As AI continues to advance, knowing what it can verify will reveal where it’s most effective.
Verifiability is becoming the clearest guide for the future of automation. And once you see it, you’ll start noticing it everywhere.