Meteuphoric

Cross posted from Overcoming Bias. Comments there.

***

I’ve been talking about how personal experimentation could be worth it for people like me, without relevant info being depleted long ago.

My next potential explanation is that people do experiment, but results aren’t aggregated and spread, so everyone has to reinvent everything.

This is exactly what you would expect in a simple model where people benefit from information, but bear a net cost from spreading it. Without incentives to contribute one’s own findings to others, there is no reason information should spread. But on closer inspection this is roughly the opposite of what the world looks like. There is a lot of advice about how to run the details of a life. Sometimes it is offered for money, but often so enthusiastically and freely as to make the most curious life-optimizer want to run away. The problem seems to be more that there is so much advice, advising pretty much the full range of behavior. There are apparently incentives for spreading such ‘information’, but not incentives to actually find any information to begin with.

This is doubly puzzling. It’s not surprising if all the possible self-help books exist. But for folks volunteering their own time to tell me about whatever relaxation technique or diet, spreading random misinformation seems low value. And again we have the question of why it wasn’t worth it, for their own benefit, to get some actual information to begin with.

A plausible explanation to me for both of these things is that just about any random innocuous change makes life seem better, and people are genuinely trying to be helpful by telling others about such ‘discoveries’. So the problem then would be widespread use of informal data collection, which is much more unreliable than people think. In which case, my own experimentation is just as likely to fail if I rely on such data collection. Experimentation in general would not be as useful as suspected – continually experimenting would make you feel like things were good, but none of your efforts would have long term payoffs.

This leaves the questions of whether and why people would be misinformed about their abilities to casually collect information about the effects of interventions on their lives. What say you?

Cross posted from Overcoming Bias. Comments there.

***

Another way personal experimentation might be worth it for me, yet not used up by those before me: there is so much innovation that there are constantly new things to test, even if people experiment a lot. Beeminder and Workflowy are new. The abilities to prompt yourself to do things with a mobile phone or eat Japanese food or use your computer in a vast number of ways are relatively new.

I doubt this explains much. The question applies to many things that have been around and not that different for a long time, e.g. wheat, motivation, reading, romantic arrangements. And even if Beeminder is new, many of the basic ideas must be old (e.g. ‘don’t break the chain‘). As a society we don’t seem to have a much better idea of the effects of wheat on a person than we do of Beeminder.

Another way innovation could explain the puzzle is if all kinds of innovations change the value of all kinds of ancient things e.g. prevalence of internet use changes the effects of going to bed early or sitting in a certain way or doing something with your hair or knowing a lot of stories. If this is the case, experimentation is worth less than it seems, as the results will soon be out of date. So this goes under the heading ‘I’m wrong: experimentation isn’t worth it’, which would explain the puzzle, except the bit where everyone else perceives this and knows not to bother, and I don’t. I will get back to explanations of this form later.

Cross posted from Overcoming Bias. Comments there.

***

I asked in what kind of world personal experimentation could seem worth it for me yet not already exhausted. Today I’ll look at one potential explanation, popular with commenters last time: I and my friends are weird in some way that causes to benefit more than usual from experimentation. Several of the suggestions below were seconded by commenters.

Robin makes a plausible suggestion in this realm: in general people do quite well by copying the right other people’s behavior in some kind of clever, intuitive, context specific way. Nerds are terrible at this though (either because they fail to copy at the outset, or because they can’t do the social interpretation necessary to correctly generalize). So they have the choice to copy other people badly, or try to reinvent a lot of things from scratch. So experimentation is much more useful for nerds. Coupled with the premise that I’m a nerd, this explains the observations and has some intuitive appeal.

If something like this is true, there seem to me to be traits beyond lack of copying skill that incline nerds toward working such things out from scratch. In general if you are already unusual on many axes, copying others on a particular one is less good, so you will have to figure things out for yourself more. Once you have determined to sleep in the daytime and practice radical honesty, the usual answers about how to improve your mood or attract a partner may not apply as well. Nerds are also more likely to have the quantitative skills to do experiments well. And nerds seem more unsettled by adherence to traditions handed to them without explanation or instructions.

These things might explain enthusiasm for explicit experimentation and innovation, but the reasons experimentation seems worth it didn’t make reference to enthusiasm. Non-nerds may copy one another fine, but there seem to be better things to do than copying. It could also be that experimentation is not worthwhile, and nerds just tend to over-rate it. But fits nicely into a category to be explored later: ‘I’m wrong’.

Another relevant way I and my friends might be weird is that we live so late in history, and in such a rich world. Perhaps it has only recently become cheap enough to track such experimentation usefully. This seems important for the more elaborate data-tracking kinds of experiments. But it seems like you can do a lot with a pen and paper, and maybe a calculator and a coin. Also, as Robin points out, there are more people to copy now, so the ‘experiment little’ path is also easier. Arguably, I say.

Another way I am strange is in being relatively young. Youth clearly makes experimentation more valuable. However I feel like it is valuable enough, and that the gains are soon enough, that I would want to do it if I were similar to myself apart from having thirty years less to live. It could be that older people are unlike me however, in that they have learned a lot more by experimentation when they were young. Is this so? It’s not clear to me.

None of these explanations seem that great. Are there other ways I’m weirdly good at benefitting from experimentation?

Cross posted from Overcoming Bias. Comments there.

***

I experiment with many things, as do those around me. Some of this is randomization and explicit records, more is just trying different things, muttering ‘VOI‘ and repeating what felt good. I refer here to everything between a cautious banana-mustard-ham sandwich and polyamory.

Robin has suggested that I over-invest in such exploration. That most new things should be bad, and so most experimentation a private loss for public gain. What’s more, there shouldn’t be lots of low hanging fruit in trying things out. Most of the things humans frequently want to do (eat, sleep, change moods, organize time, learn, interact with others) should have been well figured out in ancient times. And anything that does still need checking out should be divided between many people.

Nonetheless, it looks to me like experimentation is worth it. Lots of the things we do seem barely satisfactory, there seem likely to be better alternatives, it seems hard to learn what has been tried for what ends, or what is good from listening to others or reading, and I and my friends seem to actually find good things by looking. e.g. Beeminder, explicit charity evaluation, unusual degrees of honesty, workflowy and explicit organization seem to often add value over the defaults, not to mention many tiny things, like banana-mustard-ham sandwiches.

If it is true that a lot of experimentation is worth it, we have a slight puzzle: if there is valuable information I might glean by experimentation, why hasn’t it been worth it for others in the past to collect it and put it where I can see it?

I will try to answer this over the next few posts. Before that, what do you think?

Cross posted from Overcoming Bias. Comments there.

***

Centurion: Where is Brian of Nazareth?
Brian: You sanctimonious bastards!
Centurion: I have an order for his release!
Brian: You stupid bastards!
Mr. Cheeky: Uh, I’m Brian of Nazareth.
Brian: What?
Mr. Cheeky: Yeah, I – I – I’m Brian of Nazareth.
Centurion: Take him down!
Brian: I’m Brian of Nazareth!
Victim #1: Eh, I’m Brian!
Mr. Big Nose: I’m Brian!
Victim #2: Look, I’m Brian!
Brian: I’m Brian!
Victims: I’m Brian!
Gregory: I’m Brian, and so’s my wife!

– Monty Python’s Life of Brian

It’s easy for everyone to claim to be Brian. What Brian (and those who wish to identify him) need is a costly signal: an action that’s only worth doing if you are Brian, given that anyone who does the act will be released. In Brian’s life-or-death situation it is pretty hard to arrange such a thing. But in many other situations, costly signals can be found. An unprotected posture can be a costly signal of confidence in your own fighting ability, if this handicap is small for a competent fighter but dangerous for a bad fighter. College can act as a costly signal of diligence, if lazy, disorganized people who don’t care for the future would find attending college too big a cost for the improved job prospects.

A situation requires costly signaling when one party wishes to treat two types of people differently, but both types of people want to be treated in the better way. An analogous way to think of this as a game is that Nature decides between A or -A, then the sender looks at Nature’s choice, and gives a signal to the receiver, B or -B. Then the receiver takes an action, C or -C. The sender always wants the receiver to do C, but the receiver wants to do C if A and -C if -A. To stop the sender from lying, you can modify the costs to the sender of B and -B.

Suppose instead that the sender and the receiver perfectly agreed: either both wanted C always, or both wanted C if A and -C if -A. Then the players can communicate perfectly well even if all of the signals are costless – the sender has every reason to tell the receiver the truth.

If players can have these two kinds of preferences, and you have two players, these are the two kinds of signaling equilibria you can have (if the receiver always wants C, then he doesn’t listen to signals anyway).

Most of the communication in society involves far more than two players. But you might suppose it can be basically decomposed into two player games. That is, if two players who talk to each other both want C iff A, you might suppose they can communicate costlessly, regardless of who the first got the message from and where the message goes to. If the first one always wants C, you might expect costly signaling. If the second does, you might expect the message to be unable to pass that part in the chain. This modularity is important, because we mostly want to model little bits of big communication networks using simple models.

Surprisingly, this is not how signaling pairs fit together. To see this, consider the simplest more complicated case: a string of three players, playing Chinese Whispers. Nature chooses, the sender sees and tells an intermediary, who tells a receiver, who acts. Suppose the sender and the intermediary both always want C, while the receiver wants to act appropriately to Nature’s choice. By the above modular thesis, there will be a signaling equilibrium where the first two players talk honestly for free, and the second and third use costly signals between them.

Suppose everyone is following this strategy: the sender tells the intermediary whatever she sees, and the intermediary also tells the receiver honestly, because when he would like to lie the signal to do so is too expensive. Suppose you are the sender, and looking at Nature you see -A. You know that the other players follow the above strategy. So if you tell the intermediary -A, he will transmit this to the receiver, though he would rather not modulo signal prices. And that’s too bad for you, because you want C.

Suppose instead you lie and say A. Then the intermediary will pay the cost to send this message to the receiver, since he assumes you too are following the above set of strategies. Then the receiver will do what you want: C. So of course you lie to the intermediary, and send the message you want with all the signaling costs of doing so accruing to the intermediary. Your values were aligned with his before taking into account signaling costs, but now they are so out of line you can’t talk to each other at all. Given that you behave this way, he will quickly stop listening to you. There is no signaling equilibrium here.

In fact to get the sender to communicate honestly with the intermediary, you need the signals between the sender and the intermediary to be costly too. Just as costly as the ones between the intermediary and the receiver, assuming the other payoffs involved are the same for each of them. So if you add an honest signaling game before a costly signaling game, you get something that looks like two costly signaling games.

For example, take a simple model where scientists observe results, and tell journalists, who tell the public. The scientist and the journalist might want the public to be excited regardless of the results, whereas the public might want to keep their excitement for exciting results. In order for journalists who have exciting news to communicate it to the public, they need to find a way of sending signals that can’t be cheaply imitated by the unlucky journalists. However now that the journalists are effectively honest, scientists have reason to misrepresent results to them. So before information can pass through the whole chain, the scientists need to use costly signals too.

If you have an arbitrarily long chain of people talking to each other in this way, with any combination of these two payoff functions among the intermediaries, everyone who starts off always wanting C must face costly signals, of the same size as if they were in an isolated two player signaling game. Everyone who wants C iff A can communicate for free. It doesn’t matter whether communicating pairs are cooperative or not, before signaling costs. So for instance a whole string of people who apparently agree with one another can end up using costly signals to communicate because the very last one talks to someone who will act according to the state of the world.

So such things are not modular in the way you might first expect, though they are easily predicted by other simple rules. I’m not sure what happens in more complicated networks than strings. The aforementioned results might influence how networks form, since in practice it should be effectively cheaper overall to direct information through smaller numbers of people with the wrong type of payoffs. Anyway, this is something I’ve been working on lately. More here.