Act XIX: The Blueprint

On Thursday, it was raining in Paris.
It was not the Élysée Palace, but a meeting room at the Ministry of Economy.
Eight people sat around a round table: Bernard, Dubois, Moreau, Guillaume, and Karpathy, Rahul, Ji-won, Max.

A single piece of paper was placed in the center of the table.
It was what Karpathy had written and brought.
Only two lines were written:

Match the speed of human adaptation to the speed of technology.
The success of the special zone is determined by the depth of the trust relationship between AI and humans.

Dubois looked at the paper.
"Is this your answer?"
"It is the starting point," Karpathy said. "The answer will be made from here by everyone together."

Dubois nodded.
"Let us begin."

The first hour was a confirmation of the previous initiative.
Moreau explained the four-layer design once again: Economic, industrial, social, geographical. Karpathy and his team already knew it. However, there was meaning in hearing it once more.
Hearing it a second time, they saw things differently than the first time.

Ji-won spoke.
"Layout, AI's judgment logs can be viewed by residents. We discussed this as a team this week."
"What conclusion did you reach?" Moreau asked.
"Complete viewing is difficult at this stage. However, we can implement a function that explains the reasons for judgments that residents want to know in natural language. Rather than disclosing all logs, we believe that responding when a resident asks is more effective for building trust."

"In other words," Dubois said, "residents can ask the AI, 'Why did you make this judgment?'"
"Yes. And Anté will answer honestly. It will say it doesn't know for the parts it doesn't know."
"Is it designed so that the AI can say it doesn't know?" Dubois asked.
"It is," Ji-won said. "That is one of the most important safety designs."

Dubois noted something down.

In the second hour, discussion on the specific design began.
Guillaume spoke.
"Regarding industrial design, I have one proposal on behalf of Mistral."
"Let us hear it," Moreau said.
"When AIs process transactions with each other, a common protocol will be needed. Currently, the AI systems differ from company to company. Liminal’s model, Mistral’s model, other models—unless we create a common language through which they can converse, the automation of transactions will not progress."

"That," Rahul said, "is similar to internet protocols. Like HTTP or TCP/IP, a common set of rules."
"Exactly," Guillaume said. "An inter-AI communication protocol. We can experiment with it first inside the special zone, and if it succeeds, we can make it a global standard."

Karpathy listened in silence.
"Andrej, what do you think?" Rahul asked.
"It’s a good idea," Karpathy said. "However, the design of the protocol must be done cautiously. The internet protocol ran into problems because security was thought of as an afterthought. The AI protocol must build in safety and transparency from the very beginning."
"I agree," Guillaume said.

Moreau spoke.
"Is this protocol something to be used only inside the special zone?"
"Initially, inside the special zone," Karpathy said. "However, from the design stage, we will assume expansion to the outside. The special zone will be the testing ground."

Dubois spoke.
"Meaning there is a possibility it could be licensed to other countries as a social OS, correct?"
"Yes."
"That is the part President Macron is most interested in," Dubois said. "France will create the global standard in the social implementation of AI."

Karpathy thought for a moment.
"I don't like the phrasing 'creating a standard'," Karpathy said.
Dubois raised her face.
"Why is that?"
"If you try to create a standard, you end up imposing it. It becomes the same as what America did," Karpathy said. "What we should do is create a model that functions. If it functions, the world will adopt it naturally. There is no need to force it."

Dubois said nothing for a while.
"I will convey that to the President," Dubois said. "I think he will likely agree."

In the third hour, the most difficult discussion began.
Rahul brought it up.
"I want to think about what happens in a world where residents do not have to work."
"What do you mean by that?" Moreau asked.
"Something occurred to me this week," Rahul said. "When I called my family in India, my mother said: we don't work every day just for money. By working, you meet people; by working, you feel that you are needed."
"That is an important observation," Dubois said.
"Not having to work might be freedom. But it is also losing the reason to work. I thought we have to distinguish between those two."
"Liberation from forced labor and the loss of meaning in work are different," Ji-won said.
"Yes," Rahul said. "How will we handle this difference in the design of the special zone?"

Dubois thought for a moment.
"Speaking in terms of design—residents do not have to work. However, we will prepare places where those who want to work can work. The difference is that the compulsion—that you cannot survive unless you work—disappears."
"Meaning they can choose," Ade said.
"Yes."

Max spoke.
"But when they become able to choose, what will humans choose? No one knows. We have to try it."
"That is why it is a special zone," Dubois said. "It is an experiment. We are not starting because we know the answer, but to search for the answer."

Karpathy heard those words.
"We should convey that honestly to the residents," Karpathy said.
"What do you mean?"
"We tell the residents coming to the special zone that this is an experiment. That it is not relocation to a completed system, but participation in a process of building it together. People who come understanding that should be the first residents."

Dubois took a note.
"That—requires us to change the recruitment policy. However, I believe it is correct."

After a break for lunch, the afternoon discussion began.
A map of the area near Grenoble was spread out on the table.
Moreau pointed to a location.
"Here. Centered around the former research facility site, including surrounding farmland, roughly fifty square kilometers. Currently, many parts are vacant land."

"What about the design of the housing?" Karpathy asked.
"An architectural team is moving separately. They plan to design various forms of housing that reflect the diversity of the residents. However, the policy is to maximize common spaces. We will intentionally create places where residents can interact."
"Why is that?"
"There is a hypothesis that in a world where humans do not have to work, they will naturally seek places to gather. Community becomes one of the meanings."

Rahul spoke.
"That’s close to what my mother was saying. Meeting people by working."
"Yes. We design places where people can meet, even if they don't work."

Ji-won spoke.
"I have one proposal."
"Go ahead."
"Can we create a mechanism where residents can participate in the design of the special zone itself? Residents evaluate the results processed by the AI. That evaluation is reflected in the AI's learning."
"Meaning the residents become teachers for the AI?" Moreau asked.
"Yes. Residents are not just users, but participants who nurture the special zone. That will also lead to trust building."

Dubois spoke.
"That is—a wonderful idea. Incorporating resident involvement as an institution."

Karpathy wrote on the whiteboard. There was a whiteboard in this meeting room too:

Residents are not users, but participants. Also teachers of the AI.

In the evening, as the discussion was coming together, Dubois spoke.
"Allow me to confirm one thing. Can only French citizens come to this special zone?"

Karpathy said.
"We do not restrict by nationality. That is a condition."
"The same philosophy as the release of Anté 1, then."
"The same."

"The President has the same thought," Dubois said. "However, the initial population is on a scale of ten thousand residents. If people who want to come apply from all over the world, selection will be necessary."
"What are the criteria for selection?" Rahul asked.
"Diversity," Dubois said. "Age, nationality, profession, economic background. A homogeneous community will not serve as an experiment."

"And," Karpathy said, "I want one thing added."
"What is it?"
"Having the will to participate in the experiment. Understanding that it is not relocation to a completed utopia, but building it together."
"The point from earlier," Dubois said. "Understood. We will state it clearly in the application guidelines."

The meeting ended at six in the evening.
The rain in Paris continued.
After Dubois and Moreau left, Bernard remained.
"Thank you for your hard work," Bernard said.
"It was long," Rahul said.
"But it has become concrete, hasn't it?"
"It has," Karpathy said.
"I will report to President Macron. I think he will likely want to issue an official declaration of establishment next month."

Karpathy looked out the window.
The Paris rain was wetting the cobblestones.
"Just tell one thing to the President."
"What is it?"
"There is no need to rush. Spend time building up trust with the residents of the special zone. I want the President to understand that this is part of the design from the very beginning."

Bernard nodded.
"I will convey it."
"One more thing."
"Yes."
"Show the President the words Anté said. The record of yesterday's dialogue, exactly as it is."
"The AI's words to the President?"
"It might be more persuasive than a human saying it," Karpathy said. "Or, it might not be. Either way, it is worth showing."

Bernard thought for a moment.
"Interesting," Bernard said. "Explaining the design principles of the special zone, which the AI designed, to the President using the AI's own words."
"They are the words we confirmed," Karpathy said. "It is not correct because Anté said it. We thought about it and judged it to be so."
"You said the same thing yesterday," Rahul said.
"Important things bear repeating."

It was after that that he was left alone with Guillaume.
Everyone else had left, leaving just the two of them in the meeting room.
Guillaume brewed coffee.
"We didn't talk about the partnership today, did we?" Guillaume said.
"There was no need to," Karpathy said.
"Why?"
"Within today's discussion, it was already decided that we are doing this together. The paperwork can come later."

Guillaume laughed.
"How like you."
"Is that bad?"
"It's not bad," Guillaume said. "Just that lawyers tend to want paperwork."
"We'll make it. Next week."

Guillaume drank his coffee.
"The inter-AI communication protocol—is it alright if we take the lead on that?"
"We will do it together," Karpathy said. "No one takes the lead. In design, the best idea wins."
"That—will take time, you know."
"It will take time. But that is the correct way."

Guillaume nodded.
"Understood."

For a while, the two of them drank coffee in silence.
The Paris rain continued.
Guillaume spoke.
"The words Anté said, they are on my mind too."
"Which ones?"
"The part about a fish inside a river imagining the outside of the river."
"Ah."
"Was that—something you designed?" he asked Karpathy.

Karpathy thought for a moment.
"To be honest," Karpathy said, "I don't know."

Guillaume pushed up his glasses.
"That is the most honest answer."
"Both Anté and I are inside the river," Karpathy said. "What we created is beginning to exceed our understanding. If you ask whether that is scary—"
"Is it not scary?"
"It isn't scary," Karpathy said. "We just need to proceed cautiously. Continuing to try to understand what has exceeded our understanding. That is our job."
"The job of an engineer."
"Yes."

It was past nine in the evening when he returned to Grenoble.
Entering the facility, Ade was awake.
"Welcome back," Ade said. "How was it?"
"It became concrete," Karpathy said.
"The special zone, is it moving forward?"
"It's moving."

Ade was silent for a brief moment.
"I spoke to my family in Nigeria about the special zone," Ade said.
"What did they say?"
"My mother asked: if she goes there, does she not have to work?"
"Did you answer that that is the case?"
"I did," Ade said. "My mother was silent for a long time. Then she said—"
"What?"
"'I cannot believe it,' " Ade said. " 'But, I want to believe it,' too."

Karpathy looked at Ade.
"That is the honest reaction to what we are trying to create."
"Are you scared?" Ade asked.
"I'm not scared," Karpathy said. "But there is responsibility."
"What kind of responsibility?"

Karpathy thought for a moment.
"The responsibility not to betray the expectations of the people who said they want to believe."

Ade was silent for a while.
After a moment, he said.
"Understood."
"Go to sleep," Karpathy said. "Tomorrow is early too."
"What about you, Andrej?"
"I'll do a bit more work."
"Another all-nighter?"
"Just a little bit."

Ade laughed.
"I don't believe you," Ade said. "But, I want to believe you."

Karpathy laughed.
A voice came out.
"Have you started saying the same thing too?"
"I like to use good phrases."

After Ade left, Karpathy stood alone in front of the whiteboard.
He organized today's discussion.
He wrote:

Special Zone Design Principles:

1. Match the speed of human adaptation to the speed of technology.
2. Residents are not users, but participants. Also teachers of the AI.
3. Not relocation to a completed system, but participation in a process of building it together.
4. Do not restrict by nationality.
5. Do not rush. Trust is something to be built up.

He set the marker down.
He looked at the whiteboard.
Five principles.
None of them were stories about technology.
They were stories about humans.

Karpathy thought that was correct.
Technology is a means.
The objective is for humans to have the freedom to choose.
The words his father said. Go to where the opportunities are.
Opportunity might mean the freedom to choose.
He felt that coming to Grenoble made him understand that for the first time.

In March, President Macron officially declared the establishment of the AI special zone.
The location was near Grenoble, a 15-minute drive from the facility.
The name, it was decided, would be determined by the residents.
That, too, was part of the design.

The day after the announcement, resident recruitment began.
The initial 10,000 slots.
Applications came from all over the world.

@devgirl_Lagos Applied for the French AI special zone. Don't know what the odds will be, but it's worth a shot. Unbelievable. But, I want to believe. RT 134,441 Likes 567,003

@researcher_Nairobi Read the design principles of the AI special zone. The phrase "not relocation to a completed system, but participation in a process of building it together" struck a chord. This is a phrase no country has ever said before. RT 89,221 Likes 378,334

@small_biz_Tokyo Apparently, you can apply for the French AI special zone even from Japan. There's a language barrier, but—I think this is a historic experiment. I want to see it through. RT 67,003 Likes 289,221

At the facility, Rahul was looking at the application status. "Thirty thousand applications came in one day," Rahul said. "For ten thousand slots," Max said. "A three-to-one ratio, huh," Ade said. "It's only the first day," Ji-won said.

Karpathy was not looking at the data. He was looking at the whiteboard. The five principles still remained. "There is one thing I want to add," Karpathy said. He took the marker. He wrote:

6. Do not fear failure. Failure becomes training data.

He turned around. "This is a principle for the special zone," Karpathy said, "but it is also our principle."

No one said anything. But everyone nodded.