Pax Machina: New Institutions for Powerful AI

We have it in our power to begin the world over again.

Paine wrote this at the beginning of a great age of institutional invention: the era of constitutions, republics, and rights. Powerful AI may be opening another one, giving us both the need and the means to begin again.

By institutions, we mean the rules, roles, procedures, incentives, and organizations that structure how people coordinate. This includes courts and elections, but also venture capital, standards bodies, religious orders, professional norms associations, and basic things like contracts.

Institutions are shaped both by what participants can do, and by what the institution itself can perceive, communicate, monitor, and enforce. Powerful AI changes both. AI agents differ from humans on axes like speed, scale, auditability, replicability, and intelligence, and AI-integrated institutions will perceive and act in new ways.

As a result, some institutions will need only modest adaptations; some will become obsolete, because their underlying assumptions no longer hold; and there will be new categories of institutions, where previously binding constraints are relaxed.

This could transform the structure of society as profoundly as the shift from monarchies to representative democracies in the 18th and 19th centuries. It would reopen parts of our institutional landscape that normally seem fixed. Yet despite the growing recognition of this possibility [cites], there are remarkably few concrete institution design proposals on the table.

Most of our collective attention remains on aligning, regulating, accelerating, or slowing the AI systems themselves, while treating our surrounding institutional infrastructure as fixed. The proposals that do exist are scattered across fields, and are rarely subject to sustained public debate.

Pax Machina exists to make this work concrete and cumulative. It is a place for researchers to propose institutions suited to a world of humans and powerful AI, and to debate those designs. New governance structures will emerge whether we plan for them or not; our goal is to bring that process onto the drafting table, turning the architecture of a world with powerful AI into a deliberate, collective project.

AI changes the institutional landscape

Our political stability depends on human limitations. A president cannot personally direct every department, monitor every official, or execute every decision at once. Bureaucratic delegation and delay therefore constrain executive power as well as enableing it. But imagine a president able to command thousands of perfectly loyal AI delegates, overseeing the entire machinery of state continuously and acting far faster than courts or legislatures could respond. The words of the Constitution wouldn’t change, but some of its practical checks and balances would stop working. We might then need faster oversight, or entirely new constraints on executive capacity. Researchers have catalogued dozens of such failure modes [cite].

AI also changes what institutions themselves can do. Voting, for example, reduces a citizen’s complex worldview to a small number of multiple choice questions, partly because listening to millions of people to understand what they care about, then reconciling their competing concerns, has historically been prohibitively expensive. Recent research suggests that language models may make richer forms of large-scale deliberation possible: a model can speak with participants and help identify policy proposals that command far broader agreement than a fixed menu of options.

So, new things are possible. Some seem much worse than the status quo; some, much better. On one hand, we could have unprecedentedly responsive representation; on the other, hyper-personalized political manipulation. The same ubiquitous monitoring could finally make public institutions accountable, or turn into an airtight surveillance state. The same AI agents could either help people clarify and pursue their considered goals, or gradually displace human judgment, leaving people unable to understand or steer the systems around them.

Institutional change has happened before

Every society has an “institutional mix”: which institutions it uses, and how often it solves problems one way versus another. Many disagreements in politics are really disagreements about the ideal mix: should more problems be solved by state bureaucracies, by businesses, or by local nonprofits?

It’s sometimes easy to forget that this mix is not fixed. All the institutions we rely on in society, from corporate law and property rights to government agencies and courts, were devised by people in response to particular problems and conditions.

Some were created under frantic conditions and stress, like the post-Soviet market institutions which handed the country over to oligarchs.

But many of our most enduring institutions were designed thoughtfully through debate. A prominent example is the United States government. The founders publicly examined how their new republic might fail and built checks and balances in response. They also had a decade between the declaration of independence and the constitutional convention to learn from the states and see what worked and what didn’t. For example, Pennsylvania’s 1776 Constitution (which Paine supported, as it happens) had one legislative chamber and no governor’s veto, so whichever faction won the annual election could pass whatever it liked. Massachusetts gave itself two chambers, a governor with a veto, and independent judges, forcing political decisions through several independent checks rather than a single assembly. When the Federal Constitution was drafted, it drew on the Massachusetts architecture of divided powers.

The American Revolution was followed by a big shift in the institutional mix outside of the U.S. Similar institutions appeared in France, then in much of Latin America, then spread through the rest of Europe and much of Asia. And the 20th century brought even more shifts with the development of informational technologies.

Some worry that deliberate design can backfire (like environmental review laws designed to protect nature now blocking permits for clean energy). But if anything, this fear speaks to starting early, and giving us ample time to trial new solutions before they are direly needed.

Yet we are not approaching powerful AI with anything like this level of preparation. Concrete designs remain scarce, and we’re not experimenting with new mechanisms or building prototypes at anywhere near the scale we believe is necessary to succeed. Or at least, not yet.

What Pax Machina is for

Pax Machina is a forum for concrete proposals and debate about how institutions should work in a world with powerful AI.¹

To organize this work, we’ve developed an interactive map of the institutional landscape, which we refer to as “the grid”. Moving through the timeline shows how new institutions accumulated across different eras, organized by increasing scale (rows) and the capacities required of the participants (columns).

The historical view ends with the institutional mix we have inherited, and a second view overlays candidate institutions and research directions for a world of humans and autonomous AI agents. These entries are an initial, opinionated map of the work ahead, rather than representing any settled consensus.

Across the grid, several debates recur:

• How do we preserve human agency? How can institutions capture the benefits of AI without eroding people’s ability to understand, contest, and ultimately steer the systems they depend on?
• How can institutions adapt at AI speed without losing legitimacy? Institutions may need to respond faster as AI systems evolve. But speed can come at the expense of due process, accountability, contestability, and meaningful human consent.
• When do we adapt, and when do we invent? Existing institutions may already be suitable in some contexts, while others will need drastically new ideas.
• Centralize or decentralize? AI could make distributed coordination easier, but it could also give centralized institutions unprecedented capacity. Which functions should be concentrated, and which should be dispersed?
• What properties should institutions demand of AI systems? Alignment, cooperation, trustworthiness, and controllability suggest different relationships between agents, people, and institutions; which matter in which settings, and are there new concepts that might be more fruitful?

Pax Machina is primarily for researchers working on questions like these across mechanism design, political philosophy, social choice theory, law, economics, computer science, and AI governance. We also hope to hear from people in adjacent fields who see institutional implications in their work but have not had a natural home to develop them.

We are looking for authors with a concrete institutional proposal, a sharp objection to one, or an analysis that changes how a class of institutions should be designed. A piece might propose a new mechanism, explain how AI alters an existing institution, identify a failure mode, draw a specific design lesson from history, or offer a roadmap tied to future capability thresholds. For example, regulatory markets is a concrete institutional proposal that invites equally concrete objections, and thus we’d consider it in scope.

Our baseline requirement is specificity. Assertions that AI will upend our social infrastructure are no longer enough. We want the design details: what should change, how would it work, who holds the authority, and what could go wrong?

Responses are central to the publication. A proposal might be followed by a critique, an alternative design, or a revised version from the original author. Rather than a collection of isolated essays, we aim to produce a visible record of institutional ideas being proposed, contested, and refined.

Our window of opportunity

When the social media revolution arrived, many hoped the internet would naturally adapt to serve human connection. By the time we realized that attention markets were fundamentally distorting our public squares and democratic discourse, the incentives of the dominant platforms were already too deeply entrenched. Our window of opportunity had closed.

We cannot allow the same to happen with powerful AI.

The narrowness of the window can feel overwhelming. But consider what was built the last time one opened: the modern liberal order — constitutional democracies, open markets, human rights frameworks that let diverse societies scale up peaceful cooperation — was an act of enormous optimism, audacity, and design creativity. For all its flaws, it stands among humanity’s great accomplishments. Moments that permit work like that are rare. We are grateful to be alive at one.

Tyler Cowen, reflecting on how we must rebuild our world for strong AI, asks who among us can explain how the new pieces will fit together. Right now the answer is no one. No single person or field has a blueprint for an AI-shaped institutional order, just as no single thinker produced the liberal order. What we can do is create a public process for working it out, where concrete designs are proposed, critiqued, and revised so they are ready when we need them. This is a return to the practice of institutional design, and this is what Pax Machina is for. Let’s get to work.