The Random Room

The night I finished Taste, I had a dream. I woke in the dark and wrote it down in a hurry, before it could fade. Reading it back the next day, I saw that it was nearly the other side of that essay. The essay said that once options are cheap, the value is in the choosing. The dream asked what I would have left if the choosing were taken too.

Here is the dream first. I have tried not to improve it, only to supply the connecting words I had no time for in the middle of the night.

A room that is different every time

In the dream there is a strange room in our home. Every time I open the door, a different scene is inside: sometimes a school dormitory, sometimes an office, as if the room were switching between dimensions. Stranger still, whatever I put in stays there, and I can take things out.

I start to wonder whether something is wrong with my mind. Am I still living in the real world?

After opening it enough times, I notice a pattern. The scenes come in groups of a few, and within a group they differ only slightly. Then I see it: this is some kind of recommendation mechanism. Each group shows me a handful of near-identical versions and watches which one I interact with best.

Only when a companion points it out do I notice that I am wearing a pack, wired into my nerves. The room is something the device simulates. What I put in and what I take out are both virtual. It is just that while the device is active, I cannot touch it or feel that it is there.

Now and then there is a person in the room. That person is my partner, and only in those moments is the so-called scene the real world. Refreshing this random room over and over, picking the interaction I like best from each group, is how I make a living: I am rating for some kind of recommender. I have just been doing it too long, and my brain no longer works the way it should.

Then I woke up.

I recognized the group

What first put me on guard in the dream was not the pack. It was the pattern: a few at a time, barely different within each group. I knew it.

It is exactly the way of working I recommended in Taste: have the model produce ten versions, read and compare them, keep one and throw nine away. I called that step the choice, and said it was what still carries a price once options are cheap.

Seen from another side, the same motion is a different business. Ranking a set of close candidates and marking the best one is what people who train models call preference data. In a recommender, every click, every pause, every swipe away is a vote over a set of candidates. The motion is identical. The only difference is where the choice ends up: in something that carries my name, or as one more row in a table for the next training run.

The dream took that difference away. I thought I was choosing rooms. I was producing labels.

Who raises the waterline

There is a passage in Taste I wrote lightly. I granted that models choose too, and better every year, but said that only meant the waterline was rising, and the value would move up a layer: setting the model's standard, deciding which of its suggestions to reject, putting a name on the result. "The waterline keeps rising, but there is always a waterline."

The dream supplied the subject that sentence left out. Who raises the waterline?

The people choosing, group after group. Every pick a model learns was once someone's pick. I said taste is valuable because it rises above the default. But a recorded instance of rising above the default is the best possible material for training the next default. So taste has an odd property: it starts being harvested the moment it is used. The more carefully you choose, the sooner your choices become everyone's starting point, and the sooner the layer you stand on goes under.

That is the fear I woke up with, put plainly: that my taste is just more material to train on. If my choices can be learned, and I understand myself as the choices I make, then what gets learned is not a craft. It is me.

You cannot feel the pack while it is on

Looking back, the detail that unsettles me most is not that the room was fake. It is that I could not feel the pack while it was active.

A well-made interface is supposed to disappear. But that also means someone choosing inside one sincerely feels free. What they cannot see is who picked this group of candidates, along which dimension the differences run, and that the genuinely large questions — should this room exist at all, should I open this door — are never among the options. In the dream I did not find the pack myself; a companion had to tell me. From inside an interface it is hard to see the interface. Someone has to be standing outside.

In Taste I wrote that a choice needs a standard, and a standard comes from a purpose. In the room I had preferences but no purpose; the purpose sat on the system's side. All I supplied was which of a few small, pre-drawn differences looked better. That is not taste. That is rating.

Set it against the forms that essay listed and the gap is plain. Omission: I cannot decide that a scene should not appear, only pick among those that do. Done: the room never stops; there is always another group. A name: labels have no author, and need none. What is left is the thinnest layer, comparison. Shave taste down to comparison and it really can be learned, because it is no longer a person's judgement, only a person's reaction.

She is not an option

The worst part of the dream is my partner appearing in the room.

Not because she appears inside the simulation, but because the real world turns up as one candidate in a group, lined up beside the dormitory and the office, waiting for my score. I had been at it too long to tell which door had something real behind it.

In Taste I defined an option as a unit of supply the next unit can replace. Read backwards, that definition draws a boundary: some things matter precisely because nothing can replace them. A person, a relationship, a moment that exists only because you were there — none of these is an option. Putting them in a group to be compared is already a kind of damage.

I think that is what the dream meant by a brain that no longer works the way it should. Choose among candidates long enough and you slowly lose the category of the non-option. Everything starts to look like one of a set, including the things that were never meant to be compared.

What is left of the self

After waking I thought about it for a long time. If taste can be learned, where is the self?

I don't want to answer by consoling myself. Preferences can be learned, and the copy will keep getting closer. A model that has read enough of my choices could pick "the one I would pick" more consistently than I do. If a self is only a preference function, there is no defending it.

But the longer I thought, the less preference looked like the hard part of a self. What can be learned is which candidate I would pick from a set. What cannot is which one I am willing to answer for, and what I refuse to put among the candidates at all. The first is a distribution; the second is a string of commitments. A distribution can be copied. A commitment cannot, because its whole meaning is that a particular person carries it, and carries it through time. A model can pick the kind of person I would love; it cannot love a particular person for me. It can write the kind of essay I would sign; the consequences of signing land only on me.

That is also why, of those forms, a name is the one nothing replaces. It is not itself a choice. It is finding someone who will hurt if the choice is wrong.

Being learned from is nothing new, of course. Apprentices learn a master's hand, readers learn a writer's sentences; taste has always travelled by being imitated. What I fear is not being learned. It is the manner of it: no name, no one on the other side, only a table that keeps growing, and me inside it, not knowing what I am doing.

After waking

I am not going to stop using models because of a dream. Generating more and comparing more is still a good way to work. But I want to keep a few lines for myself.

Know that I am wearing the pack. When I choose from a set of candidates, at least ask who supplied them, along what the differences run, and whether a larger option is missing altogether.

Keep some choices outside the room. Not every judgement has to pass through candidates, comparison, and a score. Some things I would rather decide from scratch, even if it is slower, even if the result is less polished than the model's average.

Know which things are options and which are not. This one matters most. The person who appears in the room is not in any group.