A Rehearsal for Policy

Every so often there is another story like it. A policy launched with good intentions ends up somewhere near the opposite of where it was aimed. The people it meant to protect are squeezed harder; the behavior it meant to curb carries on in a quieter form. Afterwards, the verdict tends to be the same: the people who wrote it did not understand human nature.

I only half agree. The misunderstanding is real, but it is hard to pin on any one person's ability. A policy lands on thousands of kinds of people in thousands of situations, and each of them redoes the arithmetic for their own interest. No single mind can do all of that arithmetic in advance. Lately I keep wondering whether an AI could do a first pass of it.

The idea is this: build a policy simulation platform in which models play the people a policy touches — employers, workers, landlords, tenants, brokers, front-line officials — each pursuing their own interest under the new rule, and watch what happens. Before a policy reaches the real world, rehearse it in a simulated one, find the loopholes and the backfires early, and go back and revise the text.

This essay tries to set the idea out properly: where the problem really lies, why the existing tools fall short, which piece AI can supply, what such a platform might look like, and where it is most likely to go wrong.

An old story about good intentions

Economics classes like to tell the story of the cobra effect. Under British rule, the story goes, Delhi had too many cobras, so the government paid a bounty for dead ones. It worked at first, until people began breeding cobras for the bounty. When the government found out and cancelled it, the breeders released their now-worthless snakes, and there were more cobras than before. The details may not survive scrutiny, but the story travels because things like it keep happening.

In 1989 Mexico City tried to cut air pollution with Hoy No Circula: depending on the last digit of its plate, each car stayed off the road one weekday a week. The plan assumed a fifth fewer cars. Later research found no clear improvement in air quality. Many households bought a second car to get around the ban, and to save money it was often an older, dirtier one.

Many American jurisdictions adopted Ban the Box, which stops employers from asking about criminal records early in hiring, so that people with a record get a fair shot at an interview. Studies found that employers did not drop their worry once the question was gone; they guessed from race instead. Callbacks for young Black men as a whole fell, including the many who had no record at all.

After San Francisco extended rent control to more buildings, researchers found that affected landlords converted, sold, or moved into their units until the rental supply of those buildings fell by about 15 percent, pushing up rents across the city. The tenants already inside gained. The tenants who came later paid for it.

China has its own examples. Purchase limits on housing in some cities produced fake divorces, couples splitting on paper to gain a second buyer's quota and remarrying afterwards. When the labor contract law promised an open-ended contract after ten years of continuous service, some employers had long-serving staff "voluntarily resign" and re-sign before the law took effect, resetting their tenure to zero. And more commonly still: longer maternity leave was meant to protect women, and some employers responded by quietly avoiding hiring women of childbearing age.

The problem is not malice but the model

What these stories share is not ill will on the part of the people who wrote the rules. It is that the model in their heads was too simple.

Drafting a rule, people usually picture a typical person who reads the rule and does what it intends. Real people are not like that. Their circumstances vary, their resources and information vary enormously, and they respond to rules: they treat a new rule as a problem to solve, and solve it for their own advantage. The text says "do not do A." What people read is "what else is there besides A?"

Economics has familiar names for this. Goodhart's law: when a measure becomes a target, it stops being a good measure. The Lucas critique: behavior shifts with policy, so data gathered under the old rules is a poor guide to the effects of new ones. They say the same thing. A policy is not a force applied to a still object. It is a move played into a system that answers back.

There is a harder pattern too. When the rules change, the first to adjust, and the best at it, are usually those with the most information, the most resources, and the most organization. Large firms have lawyers, landlords have agents, the wealthy have advisers. The people a policy means to protect are often exactly the ones least able to play the game. So a protective rule, after every side has adjusted, sees its costs passed along layer by layer until they land back on the people it was protecting. That is close to the typical shape of a policy that backfires.

Why the existing tools fall short

It is not as if no one has tried to test policy in advance. The usual approaches each have a weakness.

Pilots. Try it small, scale it if it works. This is the closest thing to a real test, but it is slow, its costs are real, and people in a pilot know they are in one; officials try harder, and behavior after rollout may differ. Worse, many loopholes only appear once a rule has settled and the workarounds have spread, which a short pilot never sees.

Public consultation. Publish a draft, collect comments. The trouble is who comments. Those who can write a weighty submission are mostly organized interests; the ordinary people most affected either never hear of it or cannot put their case. And almost no one writes "here is how I plan to get around this."

Econometric models. Estimate effects from historical data. They are good at "what happens on average" and poor at "how will someone route around it," and they run straight into the Lucas critique: for a rule that has never existed, history holds no answer.

Traditional agent-based simulation. Simulating many interacting individuals in software is already close to what I want. But traditionally each agent's behavior has to be written by hand: "move out if rent exceeds a third of income." Such a model can reproduce the behavior someone wrote into it. It cannot come up with behavior no one thought of, and a loophole is precisely the behavior no one thought of.

The piece AI can supply

What a large language model changes is that it can be asked to play a particular person, read a rule with that person's circumstances, goals, and common sense, and reason out what they might do. Its behavior does not have to be scripted in advance. It works things out.

That matters a great deal for policy. Tell a model playing the owner of a small restaurant that "from next month, firms with ten or more staff pay an extra levy," and with no prompting it may think of keeping headcount at nine, moving some people to contractors, or splitting into two separately registered shops. That is exactly what drafters miss most easily: not someone breaking the law, but someone finding, inside the law, a path the drafters never pictured.

Researchers have started down this road. In 2023 a Stanford team built a small town of two dozen or so model-driven characters, who organized a party and passed news along on their own. A later study built an agent for each of more than a thousand real people from interviews with them, and those agents answered a social survey nearly as consistently as the people themselves did when retaking it two weeks later. Economists have begun discussing models as simulated subjects that reproduce classic behavioral experiments. Earlier still, an "AI Economist" used reinforcement learning to search for tax policies.

All of this is a long way from a platform that can test real policy. But it shows one thing: having models play different people, and produce behavior in interaction that no one scripted, is no longer fantasy.

What a rehearsal platform might look like

To make the idea more concrete, here are the parts I have in mind.

Parsing the policy. First, break the text into rules that can be executed: who is covered, what is required, what the rewards and penalties are, how eligibility is judged, who enforces it. The breaking-down is valuable in itself. The vague places, the ones that admit more than one reading, are usually where the loopholes will be, and should be flagged first.

A population of identities. From census, survey, and industry data, build a population as close to the real distribution as possible. Each identity is not a label but a situation: income and assets, dependants, sources of information, appetite for risk, the resources at hand and the constraints in force. Beyond the people a policy targets directly, include three kinds of actor that are often left out. Enforcers, because front-line officials bend a rule to hit their own targets. Intermediaries, the people who study rules for a living and sell the workarounds, usually the first to find a loophole and the ones who spread it. And bystanders, whom the policy never mentions but onto whom its costs may be shifted.

Environment and interaction. No one decides alone. A landlord's choices change the market a tenant faces; a trick one person finds travels through a social network to many more; officials who see everyone taking the detour loosen or tighten. So the simulation should run for many rounds and let behavior evolve through interaction. Only then do second- and third-order effects show up, instead of a simple sum of everyone's first reaction.

A red team. Besides simulating what ordinary people would do, add a set of agents whose only job is to maximize their own advantage within the rules and hunt for every loophole. This borrows from security: before a system goes live, you pay people to attack it. A policy deserves a penetration test too.

Evaluation. The output should not be a single score. At minimum it should answer: were the policy's goals met; on whom did the gains and costs fall, and in particular did the people it meant to protect end up better or worse off; what unexpected behaviors appeared, ranked by likelihood and harm; how costly is enforcement, and do enforcers have reasons to bend it.

Iteration. Revise the text, run it again, compare versions. The real value of the platform is not scoring a policy. It is making "change one clause, try again" cheap enough to do dozens of times before anything is published.

Walking through an example

Take longer maternity leave. Suppose the draft extends it from 98 days to 180, with employers continuing to pay full wages throughout.

Put several identities into the simulation: owners and HR managers at firms of different sizes, young women who are single or married without children, women who already have children, men applying for the same jobs, a recruiting platform, and the labor inspectorate. Let them run through a few hiring seasons.

The HR agent at a large firm might say the cost is bearable, but that all else equal it will lean toward women who already have children, or toward men. The small-business owner reacts far more sharply: in a firm of a dozen people, half a year of leave means hiring a stand-in and paying two salaries for one job. He will never put a gender requirement in the job ad, but he will ask about marriage and children in the interview, or screen the résumés out before anyone gets that far. Coded filters may appear on the recruiting platform. Young women notice the shift in interviews, and some start hiding their plans. The inspectorate finds this kind of discrimination almost impossible to prove.

The evaluation might conclude: women already in work who can take the full leave are better off; young women not yet hired, or looking for work, are worse off. That is exactly what it looks like when the people a policy meant to protect are eaten into further.

Then revise and run it again. Have social maternity insurance pay for the extended leave, so employers no longer bear it directly. Or give fathers leave that cannot be transferred, so that hiring a man carries a similar "risk" and employers lose their reason to avoid women. Rerun, and check whether the small-business owner's urge to screen has weakened and whether outcomes for mothers and for women without children have evened out. None of these fixes is new; many countries adopted them long ago. The point of the platform is to let drafters see, before they commit, how the first version would hurt the very people it was written for.

Where it is most likely to go wrong

Having come this far, I should be just as clear about the weak points. A simulation that is trusted too much can do more harm than having none.

A model is not a person. A language model's "human nature" comes from the text it has read, which over-represents people who are online a lot, who write, and who speak dominant languages. Playing a migrant worker, an elderly person, or a minority, it may well produce a stereotype rather than a situation. It is also often more reasonable and more rule-abiding than real people, or else, the opposite, it plays everyone as a calculating rational actor. Real people procrastinate, avoid hassle, and do nothing because they never heard of the policy, and that inaction shapes outcomes too.

It has to be calibrated. The platform should be tested against old policies whose outcomes are known. Unprompted, can it reproduce Mexico City's second car, or the fake divorces under purchase limits? There is a trap here: the model has probably read about those outcomes in training, so reproducing them proves no foresight. The more honest test uses policies introduced, and resolved, after the model was trained.

It is a stress test, not a prophecy. I think the right role for such a platform is not to predict that "unemployment will rise 0.3 points," but to produce a list of behaviors that could plausibly appear and tell drafters where to worry. It is closer to a wind tunnel, a war game, or a security red team than to a weather forecast. Its value is in finding problems, not in proving there are none.

It must not become a rubber stamp. The most dangerous use is citing "the simulation found no problems" to defend a policy and shut down criticism. A simulation that found no loophole only shows that the simulation found none. To guard against this, the setup, the population, the prompts, and the results should all be public, so anyone can challenge them, rerun them, and add cases they know of that the platform missed.

It cannot replace real people. A simulation should tell us whom to ask and what to ask, not let us skip asking. If it shows small employers screening out women of childbearing age, the next step is to go and talk with small employers and with women looking for work. Above all the people a policy means to protect, whose voices are the easiest for statistics and models to flatten.

Anyone can use the same tool. A platform that helps drafters find loopholes can help interested parties find them sooner. That worries me less than it might. Professional loophole-hunting is something interested parties have always done, and done well. What is missing today is the same capability on the side of the people writing the rules. The platform puts drafters on at least the same starting line as the people best at playing the game.

Rehearse before the show

Whoever writes a rule imagines people living under it, while the real people turn around and imagine the rule. That gap used to be filled with experience, pilots, and repairs after the fact, and the bill was usually paid by those least able to play the game.

AI will not make policy perfect, and it should not decide for anyone. But for the first time it lets us walk a policy, before it lands, through thousands of simulated people each with their own designs: to see who takes the detour, where the costs finally settle, and whether the people it meant to protect have been pushed further to the edge again.

Every play is rehearsed before opening night. Rules that shape millions of lives deserve a rehearsal too.