Field Notes

Important Problems

From You and Your Research by Richard Hamming:

"You will not do important work if you do not work on important problems."
Great scientists have 10 to 20 important problems in their field, always looking for an attack.
An attack means they must have an angle.
They are consistently mapping new ideas to these problems.

What are the problems of our field?

The hope is that 100 different perspectives could explain the distribution better than anyone could on their own.

List So Far

An agent that is constantly improving, where the fail/correct signal comes from user behavior.
J: a more accurate name could be "continual training"; models are not learning right now, they are being trained.
F: this might be an engineering problem at this point.
Is this one of the biggest problems? Maybe not in the way I am framing it, because it seems like there is a clear attack and direction.
This might jump into the category of self-evolving, hyper-personalized systems.
For some people there is no distinction between this and generalization.

If a model gets trained in an Amazon environment, why does it need to use one for eBay?
A model should not even need to see an Amazon environment.
Given the foundations of a discipline or knowledge, it should be able to move up the stack and build its own trees of knowledge.
Maybe not even the foundations. If you get put in an environment, would a human be able to learn to use a computer by himself?
Maybe I am confusing intelligence with Francois Chollet's definition: efficiency at skill acquisition.
Ilya defined it here in a more poetic way, but seems to call it "continual learning."

Current architectures require huge amounts of data to be useful.
Humans are far more sample efficient. With much less data, we learn things in detail and develop intuitive understanding of fields.

Hamming defines creativity as usefully putting together things that were not previously perceived as related, and says the initial psychological distance between the two may matter most. This is not the same as originality.
If a single human had access to this much knowledge, they should be able to come up with very creative solutions, connecting biology and motorcycles, or spaceships and gardening. Why cannot models?
What does it even mean to have intuition or to connect the dots?

What is a value function? Ilya defines it here.
As we expand horizons and take hundreds of steps, and continue having sparse completion signals, can we terminate rollouts early and sense directional correctness?
Michael and John mention this here.
"Value functions are extremely robust in people." A teenager can learn to drive in 16 hours, and there is no verifiable reward.

Even with so many companies in the space, we still lack a deep understanding of what good agentic memory means.
There are architecture and task tradeoffs that still feel underexplored.
This seems like it should be solved and expanded in 2026.

I do not have a take here yet: no clear example and no clear attack.
I like the notion of an evolutionary process or algorithm, and biology.
I want to learn more about neuroscience. What are we missing? Ilya has highlighted this multiple times.
The notion of self-play training is interesting.
F: maybe the current paradigm works well if labs spend $10B more on RL environments, and they are going to do it.
Ilya: RL makes the model focus too narrowly.
Ilya: human neurons might do much more compute than we think.
Complexity and robustness trade off. Simple things are broadly useful. Emotions are simple.

Evals have been purely academic over the past five years. We are only now starting to connect dollars as a proxy for agent capabilities.
A: labs' ARR as a proxy. I disagree. With so much money floating in the space, ARR is not directly correlated. Maybe overall company growth vs employee ratio and layoffs.

K: this might actually be the problem.
In a world where humans do not perform any task better or faster than AI, what is our purpose, what is the hierarchy, and who owns all the power?
Maybe the person reading this, in the top 0.1%, living in the Bay, will be okay and even benefit.
L: the best thing you can try to do is own a small piece of an infinitely growing entity, like an AI lab.
We should see gigantic leaps in longevity. What does the life of a 500-year being look like?

In the current paradigm, we do not really know what downstream behavior comes from which section of the vast corpus we embed.
Mechanistic interpretability is in its infancy.

HCI means human-computer interaction.
We are building systems intended to outperform humans at everything, instead of systems that together with humans achieve 10x more.
This might be too late, and we may already be on the path of self-deprecation.
Also: is there nothing better than a chat interface?

How can the average individual be protected against a bad actor using Opus 6, considering the average individual outside the Bay barely knows what Anthropic is?
Not only the individual, but humanity.
A: can you maintain alignment during recursive self-improvement?

Very connected to creativity and intuition. Everything else seems like engineering.
Producing new ideas, implementing them, and running a never-ending self-improving loop.
In a separate direction, our ETA for new drugs to market could become instant, from personalized medicine to deep access to your health.

Robotics, FSD, VR, and game generation all require spatial generation and understanding.
The future requires world models.

Removed from the original list:

Ilya said in Dwarkesh something like: post-training is the only thing that differentiates labs; pre-training is the same. There are too many degrees of freedom.
Right now only Anthropic and a few environment companies seem to know what works and how well.
But there are more than 50 of these companies.
Either it is a skill issue or just a matter of time. We will have a clear recipe by 2026 for sure.