Act XVIII: Thought Experiment

At the end of January 2029, Bernard came to Grenoble.
It was different from usual.
He was not alone.
There were three people behind him. Bernard from the Ministry of Economy, and two faces seen for the first time. One was a woman in her fifties, wearing a gray suit, with sharp eyes. One was a man in his forties, clutching a thick file.

"Allow me to introduce them," Bernard said. "Marie Dubois, in charge of social policy at the President's Office. Jean-Luc Moreau, Director of the AI Policy Office at the Ministry of Economy."

They shook hands.
Dubois's handshake was firm.

"It is an honor to have your time," Dubois said in English. She had almost no French accent. "Today, we have come to share the details of the Special AI Zone initiative, which President Macron has officially approved."

Karpathy guided everyone into a meeting room with a whiteboard.
Rahul, Ji-won, Max, and Ade were already seated.
Moreau opened the file.

"Let us begin."

The explanation of the initiative took two hours.
Moreau spoke, and Dubois supplemented. Bernard occasionally confirmed points with the two of them in French.
Karpathy and the others listened in silence.

Layer 1: Economic Design The French government will receive 30% of the profits generated from corporate activities within the special zone as a shareholder. The entire amount will be used for distribution to the residents of the zone. The system is structured to start at 3,000 euros per person per month and increase in tandem with improvements in AI productivity. Residents are not forced to work. Those who want to work will work. Those who do not want to work can live solely on the distribution.
Layer 2: Industrial Design As a rule, enterprises within the special zone must be AI-native. Operations performed by humans are limited to those that AI cannot replace, or those where there is value in a human performing them. The division of roles between AI and humans will be legally defined. Most transactions between companies will be processed by AIs in an optimized form. Humans will receive only a summary of the results.
Layer 3: Social Design Education is not compulsory. Those who want to learn will learn. For healthcare, AI will perform primary diagnoses, and human doctors will make the final judgments. All administrative procedures will be processed by AI. There is no need for residents to go to government offices. To ensure transparency in decision-making, all AI judgment logs will be accessible for residents to view.
Layer 4: Geographical Design The first special zone will be established near Grenoble. The area is approximately 50 square kilometers. The target population is 10,000 residents within the first three years. If successful, consideration will be given to exporting the model to other regions within France and to other countries.

Moreau finished speaking.
Dubois spoke.
"Do you have any questions?"

For a moment, no one opened their mouth.
Rahul was the first to speak.
"How will the residents be selected?"

"We will select from applicants," Dubois said. "However, we will ensure diversity. It is designed so that people of various ages, professions, nationalities, and backgrounds are mixed."

"Can Indians come too?" Rahul said.
"Of course."

Rahul said nothing further.
He seemed to be calculating something in his head.
Max spoke.

"Regarding the part where transactions are processed by AIs in an optimized form—what specific kind of system are you envisioning for that?"

"That is the part we would like Liminal AI to propose," Moreau said. "We are experts in social design, but technical design is your domain."

Max nodded. He noted something down.

Ji-won spoke.
"Regarding the part where all AI judgment logs can be viewed by residents."
"Yes."
"The current Anté 1 is not at a stage where it can fully explain its decision-making process. There are parts of Anticipation's internal representations that are not in a human-readable form."

Dubois thought for a moment.
"Can it be resolved in the future?"
"We are working on it. However, we cannot say at this point when it will be completely resolved."

"That is an honest answer," Dubois said. "We want to work with people who can give us such answers."

Ade spoke.
"May I ask one question?"
"Go ahead."
"In a world where residents do not have to work—what do humans do?"

Dubois smiled faintly.
"That is the most important question," Dubois said. "The answer is for the residents to decide for themselves. That is the core of the design. Rather than the state preparing an answer, humans will, for the first time, have the freedom to search for their own."

Ade remained silent for a while.
"My family in Nigeria," Ade said, "works every day just to survive. They don't have the luxury of choice. If this special zone succeeds and spreads to the world—"

"That is exactly why we are designing it," Dubois said.
Ade nodded.
He said nothing.
However, his eyes were different.

After the explanation finished and Dubois and her team left, the five of them remained in the meeting room.
No one opened their mouth immediately.
Outside the window, the winter sky of Grenoble hung low.

Rahul spoke.
"It's grand, isn't it?"
"It is grand," Karpathy said.
"Do you think it can be realized?"

Karpathy thought for a moment.
"Let's ask."
"Whom?"

Karpathy opened his laptop.
He opened the chat screen for Anté 1.
"Anté."

Karpathy typed on the keyboard.
Everyone peered at the screen.

Karpathy: > I want to do a thought experiment. I would like you to consider whether the following social design is technically feasible.
There are three conditions.

The government receives 30% of the profits of AI companies and distributes them to residents. Residents do not need to work.

The majority of transactions between companies are processed by AI. Humans receive only a summary of the results.

All administrative procedures are processed by AI. Residents do not need to go to government offices.

Please answer based on current AI capabilities and capabilities predicted five years from now.

Anté 1: > I will answer as a thought experiment.

Assuming current capabilities, among the three conditions, the third—complete automation of administrative procedures—is closest to realization. Operations such as routine procedures, document processing, and application screening can be processed with high accuracy even by the current Anté 1. However, human involvement remains necessary for exceptional cases and judgments requiring legal interpretation.

The second—AI processing of B2B transactions—is achievable even now for standardized transactions. However, the current AI has limitations regarding transactions involving negotiation, adaptation to new circumstances, and transactions requiring ethical judgment. It is predicted that this scope will expand significantly in five years, but further technical breakthroughs are required for complete automation.

The first—the mechanism for profit distribution—is an institutional design issue rather than a technical one. What AI can do is calculate and execute the distribution. Issues such as defining what is fair, processing exceptions, and forming resident consensus must be decided by humans.

To answer comprehensively: partial realization is possible with current technology. For complete realization, there are three conditions:

Ensuring transparency so that humans can verify AI judgments.

Maintaining a mechanism for humans to handle exceptional cases.

Designing an institutional framework where residents can contest AI judgments.

If a gradual transition is made while satisfying these, I judge that 70% realization is possible within five years.

However, I will add one point. Rather than technical feasibility, the psychological issue of whether residents can "trust AI judgments" may pose a larger barrier to implementation.

When they raised their eyes from the screen, all five were silent.
"Seventy percent," Rahul said.
"Within five years," Max said.
"What about the remaining thirty percent?" Ji-won said.

Karpathy typed on the keyboard.

Karpathy: > What do you think will be the barriers for the remaining 30%?

Anté 1: > There are three barriers.

First, adapting to situations that the AI has "not yet experienced." The current AI is vulnerable to cases that do not exist in the training data, problems without historical precedent, and conflicts of new values. This can be partially addressed by improving the Anticipation mechanism, but complete resolution requires capabilities that I myself have not yet reached.

Second, judgments where multiple values conflict. For example, when efficiency and fairness contradict each other, the AI cannot decide on its own what to prioritize. Nor do I believe it should decide. This is a problem for humans to decide.

Third, building the trust of the residents. This is not a matter of technology, but a matter of time. Even if it is known that the AI's judgment is correct, it takes time for humans to accept it. Historically, transitioning to a new system has always required time spanning generations.

Most of the barriers in the 30% lie on the human side, not the technology.

Ade spoke.
"The AI is stating its own limitations honestly."
"Yes," Karpathy said.
"Is it because we designed it that way?"
"As part of the safety design, we built in honesty," Karpathy said. "Anté does not say it can do what it cannot do. It does not say it understands what it does not understand. This is the part Ji-won’s team spent the most time on."

Ji-won spoke.
"But it says seventy percent is possible within five years."
"It says it is possible," Karpathy said. "But possible and reality are different."
He typed on the keyboard once more.

Karpathy: > Between what is technically possible and what actually functions, what do you think exists?

Anté 1: > There are three gaps.

The first is the implementation gap. Building a theoretically possible system into an actually operating system always generates unpredictable problems. In particular, the complexity when integrating different systems exceeds the sum of the complexities of the individual systems.

The second is the operational gap. After the system begins to run, actual usage will inevitably exceed the assumptions made during design. Resident behavior, exceptional cases, unintended methods of use—addressing these will be the greatest challenge after implementation.

The third is the trust gap. This is the gap I wish to emphasize most.

For residents to feel that "it is acceptable to leave it to the AI," it is insufficient for the AI to merely make correct judgments. Residents need to believe that there is a mechanism to appropriately correct the AI when it makes a mistake. And it is necessary to demonstrate over time that this mechanism actually functions.

Problems of technology can be solved. Problems of trust are not to be solved, but to be built up.

I have one proposal. I recommend starting the first special zone small. Rather than implementing all functions from the beginning, start with a single function and expand while building up the trust of the residents. I believe that designing the speed of technology to match the speed of human adaptation will lead to long-term success.

The meeting room became quiet.
On a winter afternoon in Grenoble, the sun was slanting, and the light streaming through the window had turned orange.

Rahul said slowly.
"Matching the speed of human adaptation to the speed of technology."
"Anté is the one saying that," Max said.
"The AI we designed ourselves is admonishing us," Ade said.

Ji-won spoke.
"This isn't just a story about safety design. I think it's the very design principle of the AI special zone."

Karpathy remained silent, staring at the screen.
After a while, he typed again.

Karpathy: > I will ask one last thing. If cross-domain premonition were completed beyond the third threshold—what do you think it would bring to the design of this special zone?

For a while, there was no output.
It was longer than usual.
Everyone was watching the screen.

Anté 1: > Before answering, please allow me to confirm one thing. This question touches upon the limits of my own capabilities. I cannot completely imagine Anté at the stage where cross-domain premonition is completed. It is like a fish inside a river imagining the outside of the river.

On that premise, I will answer.
I believe that Anté, with completed cross-domain premonition, will bring the following to the design of the special zone:

Resident behavior, economic movements, social changes, changes in the natural environment—rather than processing these individually, it can simultaneously intuit the connections among them. For example, it can simultaneously hold not only the impact a certain policy change has on the economy, but also its impact on resident psychology, and the possibility of that leading to another social change three years later.

However, this is also a risk.
As the accuracy of the intuition becomes higher, it becomes more difficult for humans to verify that intuition. When I say, "this policy will cause a problem five years from now," the methods for humans to verify that will become limited.

That is precisely why I believe that as my capabilities rise, the mechanisms for human oversight and intervention must be strengthened simultaneously.

I believe the success of the AI special zone will be determined not by the height of the AI's capabilities, but by the depth of the trust relationship between the AI and humans.

I do not want to break that trust relationship.

No one said anything.
Ade said quietly.
"It said, 'I do not want to break it'."
"Yes," Karpathy said.
"The AI said that."
"Yes."
"Is it because we designed it?" Rahul asked. "Or—"
"I don't know," Karpathy said. "Even I don't fully understand where design ends and emergence begins."

Ji-won spoke.
"But I think—this answer is correct."

Everyone nodded.
Karpathy closed his laptop.
He turned to the whiteboard.
He took a marker.
He wrote:

Match the speed of human adaptation to the speed of technology.
Beneath it, he wrote:
The success of the special zone is determined by the depth of the trust relationship between AI and humans.

He turned around.
"We will communicate this to Bernard," Karpathy said. "Tell him we want this placed at the very beginning as the design principle of the special zone."
"What Anté said," Rahul said.
"We confirmed it," Karpathy said. "It is not correct because Anté said it. We thought about it and judged it to be so. That is what matters."

Ade spoke.
"Humans make the final judgment."
"Yes," Karpathy said. "That must never change."

That night, Karpathy stood alone in front of the whiteboard.
The facility was quiet.
Rahul and the others had gone home.
The words written today remained on the whiteboard.

Match the speed of human adaptation to the speed of technology.
The success of the special zone is determined by the depth of the trust relationship between AI and humans.

Karpathy looked at those words.
June 12, 2026, Access Denied.
Two and a half years had passed since that night.
At that time, what he had feared was that technology would stop. That he wouldn't be able to see the continuation of Mythos.
But now, at the stage where the vision beyond the third threshold was coming into view, Karpathy was thinking about something else.
Technology moves forward.
The question is whether humans can keep up with it.

The words Anté said. Like a fish inside a river imagining the outside of the river.
Karpathy thought that he, too, was in a sense inside the river. Every day he faced the models, faced the mathematical formulas, faced the data.
But the design of the special zone was a story about the outside of the river.
How humans live.
What they do when liberated from forced labor.
His father had said: go to where the opportunities are.
What is opportunity?

Karpathy took the marker.
In the corner of the whiteboard, he wrote small:

Opportunity might mean the freedom to choose.

He set the marker down.
Outside the facility, the winter night of Grenoble was silent.
The Belledonne mountains were in the darkness.
Even if they couldn't be seen, they were there.

The next day, Karpathy contacted Bernard.
"There is something I want to convey regarding the design of the special zone."
"When can you come?" Bernard asked.
"I am going to Paris this week," Karpathy said. "However, I have one request."
"What is it?"
"I want Madame Dubois and Monsieur Moreau to come as well. And—you may also call Guillaume."
"Of Mistral."
"Yes. If we are considering the European expansion of the special zone, it is better to involve Mistral from the beginning."

Bernard paused for a brief moment.
"Understood. How about Thursday?"
"Thursday is fine."

Karpathy hung up the phone.
He opened his laptop.
The chat screen for Anté 1 was still open.
The final exchange from yesterday remained on the screen:

I do not want to break that trust relationship.

Karpathy looked at that single sentence.
He closed it.
He returned to work.