Category Archives: 1

Suppose that you are listening to music, and you reach a song that makes you sad. How do you respond? Here are two ideas:

A) Be sad. Perhaps think about bittersweet memories. Stare into space. Get completely sidetracked and cry a bit. Downgrade your assessment of how good your life is overall. Until some happy music comes on, at least.

B) Decide if you want to be sad, and adjust the playlist accordingly. Maybe you determine that your goals would be better served by listening to patriotic music on this occasion. So you change it. Even though the sad music is now making you feel like listening to sad music.

It seems to me that these correspond to two natural classes of things people are doing when they ‘respond’ to stuff.

The first kind of response is a relatively automatic reaction to a stimulus. Feeling sad when you hear sad music. Finding a clever retort if someone is rude to you.

The second kind of response is a continued pursuit of your goals, adjusted for any information contained in the stimulus. Turning off sad music if it doesn’t seem helpful. Walking away from a rude person if talking to them is not creating a lot of value, while perhaps considering whether their criticism is relevant to you.

I’m going to call these reflexive responses and agentic responses.

More examples of stimuli and reflexive and agentic responses to them:

• Being cold: curl up under your meager blanket and shiver vigorously // get out of bed, cross the very cold room, turn on the heater.
• Hearing a joke: Laugh proportional to the humor of the joke, adjusted for offensiveness and attractiveness of the joke teller // laugh if you think it would be good to raise the status of the person telling you the joke, or to make them happy, or to make the situation less awkward, etc.
• Seeing misinformation on a blog: write a comment correcting the error // add ‘online misinformation’ to your mental list of problems in the world, and then if you decide that it is the most important one at some point, seriously scheme about what to do there.
• Someone starts a conversation with you: say the natural next thing at every juncture // decide if there is something you want to achieve by talking to the person, and then steer the conversation appropriately (perhaps toward winding up)
• Physically suffering: curl up in a ball, close your eyes and whimper // search for painkillers, make a doctor’s appointment then leave your house and go to it
• Emotionally suffering: avoid thinking about the topic, cry, go over the source of distress in your head, tell other people that you are suffering // try to figure out why you are suffering, and then stop it, even if that involves some amount of thinking about unpleasant things and having uncomfortable conversations.
• Being employed: do the things you are meant to do at jobs, perhaps hinted at by the instructions // choose the bits of the job that are relevant to your goals and emphasize those to the extent that makes sense within the bounds of not jeopardizing your job or failing at your promises.
• Being called on to give a speech: say the things that are meant to go in speeches // say the things you want the audience to hear
• Your partner being rude to you in front of your friends: disrespect them aggressively right back // infer that your partner may not respect you enough or doesn’t understand social norms or made an error, and make a mental note to figure out which and address the problem later. Decide whether it is valuable to save face in front of present company, and aggressively disrespect them right back, or be nice, or whatever, as appropriate.

For a more real example, at the time of writing most of this, I was in much pain, and was lying in bed thinking something like ‘Pain! Why pain? Why me? Ow. Pain! Pain pain pain’. Then I thought that perhaps all the thinking about pain was worse than the actual pain, and that even though noticing that I’m in pain every two seconds comes pretty naturally to me when I’m in pain, there are probably actions that better achieve my goals, if I can do them. So I decided to write a blog post instead, which to my surprised actually worked, at least for a bit. [Added upon coming back to this draft: my ability to focus on things other than being in pain didn’t last terribly long if I recall, but it was good for a bit.]

I claim that it is helpful to distinguish reflexive and agentic responses.

It seems that a key pattern in how humans interact with the world is that they notice events, and then feel the need to respond to them. Even to a person who is otherwise pretty consequentialist, it somehow seems very natural to say a thing right now for no apparent reason, except that some random person said a different thing in your vicinity, and the words you are saying are a semantically and socially natural response to the words that they said. You put your actual projects on hold, because you have to respond.

Furthermore I think when we respond, we usually do it in the reflexive style. Which is natural: there is arguably a lot of responding to be done, and we can’t think about all of it. But it is nice to remember sometimes that there is also the option of responding agentically. Responding agentically takes mental and perhaps other effort, but that aside will tend to be better (by definition).

I’ll say more about these classes of behavior as I see them, partly in the hope that this helps with drawing them so that other people know what I’m talking about.

You might think of reflexive responses as sort of based on feelings and agentic responses as sort of based on explicit thought. I think that isn’t right—in particular, there are a lot of reflexive responses that also mostly involve explicit thought. It’s just that the explicit thought isn’t about how to achieve your goals. For instance, if a person argues with you, you might have a reflexive response of constructing a counterargument and then sending it to the person. Or if you are watching the news and there is a surprising event, you might have a reflexive response of thinking about its implications, and remarking upon them verbally to your companion.

Some reflexive responses were designed by an agentic response previously. For instance, if you noticed before that you should just never listen to a particular song because it will ruin your day, then if it comes on you might skip past it near automatically.

It is often hard to not do a reflexive response. For instance, if you are angry, it can be hard (and arguably destructive) not to express it, perhaps without careful regard for the social consequences. This seems like a fine reason to react reflexively often. I still think that observing the existence of a decent alternative reaction often makes the reflexive response less naturally appealing.

Agentic responses very often involve not doing anything. Because it’s not that common that an event in your vicinity substantially alters what is the best thing for you to do next. More common than by chance, because things in your vicinity are much more relevant to you than other things. But still not that common.

I’m rolling a lot of different kinds of behavior into reflexive responses. Intuitive completion of patterns you are part of, fulfillment of roles, expression of the feelings that the stimulus makes you feel, following immediate incentive gradients, doing what feels right, fulfilling instincts.

Reflexive responses are often good because they are cheap and predictable. It is cheaper to finish a pattern or to fulfill the role than to rethink your whole plan in light of new evidence. And if people usually respond to X with Y, this perhaps makes them easier to interact with. Arguably, you can just live a whole life of one reflexive response after another, and then you don’t have to have goals at all, which potentially represents a real saving (at the cost of everything you might have wanted, if you had wanted anything).

Reflexive responses are associated with getting stuck in local optima. For instance, shivering in bed, which is less cold than getting out of bed, but more cold than turning on the heater and waiting five minutes. My guess is that they are also more associated with pathological large scale social interaction traps, such as the toxoplasma of rage. (While one might reasonably decide that reacting with outrage to something on the internet is the best way at hand of forwarding one’s goals, this has got to be a lot rarer than reacting out of anger is).

I get some value out of having these concepts. My friend has argued that they are part of a fundamentally wrong worldview, and I think he’s partly right (a discussion for another time), but I still think they are a good enough approximation of an important thing, and his better worldview doesn’t seem to naturally support a similar distinction.

Related: systems and stories, deontology and utilitarianism, reflex agents and goal based agents, thinking outside the box vs. thinking inside the box.

I visited the DC area recently and recorded a couple more podcasts with Robin Hanson while there:

• Robin’s book, Age of Em
• Rationality

Talking to Robin is pretty entertaining and intellectually invigorating, and I recommend it.

The rest of our very occasional podcast series:

• Ads
• Relations
• Play
• Saving The World
• The Big Question
• School
• Future
• Signaling
• Idealism

A truish story with all names and many circumstances altered:

Alice and Bob were friends, but their friendship had seen some rough patches recently due to an ill-advised business experiment and some ensuing uncomfortable feelings. Today they were co-working, with the intent of mending whatever was broken through dedicated inattention. The two of them sat down in Alice’s kitchen. Bob took out paper, pens, and a loud bell. He picked up the bell and proceeded to pace back and forth across the room jangling it with the vigor of two arms and staring into space.

It turned out that Alice loved the sound of loudly jangling bells, which reminded her of the beautiful church she used to live next to. And she was working on her own general solution to coordination problems—a puzzle so fascinating that she could not possibly be distracted, even by the delightful bell song. So the noise caused Alice no suffering.

However, Alice reasoned that Bob had no way of knowing that she would not be harmed, and that he really should have expected that his bell-ringing would annoy her a lot in expectation. So she had little choice but to infer that Bob didn’t care about hurting her. And that did hurt.

Furthermore, she reasoned that Bob must be aware that she would be hurt by observing that he didn’t care about hurting her. So he should have anticipated not only some suffering in expectation from the bell, but also this second level of more certain suffering from observing Bob’s indifference. Knowing that he was happy to deal her even that much suffering was even worse. And furthermore he didn’t even care about this extra suffering!

On top of that, she reasoned, once there is enough common knowledge of enough certainty of enough suffering willingly inflicted, Bob can’t be doing this by accident. It becomes an intentional affront. A message of hatred, rather than an inadvertent sign of indifference.

She became extremely angry and marched out of the front door.

(Bob made a mental note that Alice really didn’t like bell sounds.)

Knowing that someone knowingly hurt you is hurtful. And knowing that someone knowingly hurt you by indirectly causing you to know that they knowingly hurt you is hurtful. And so on.

I suggest that social injury often has this character of being magnified iteratively by approaching common knowledge.

Perfectly legitimate offense doesn’t even need to stand on the ground at all. Suppose that I like being slapped in the face. Also, I know that you know this. But I also know that you don’t know that I know that you know that I like being slapped in the face. Then you slap me in the face. I’ve got to figure you are willingly harming me with your seeming desire to harm me, even if you don’t think I will actually mind the slap per se. Alice was reasonable to be upset, even though she liked the bell sound.

I expect something similar can happen at a group level. There is an action that hurts a small fraction of some group of people. Then doing it indicates that you are fine with a chance of hurting people from that group, which hurts the feelings of the whole group, and causes enmity with whatever groups you are saliently a member of. Then if people continue to do the action, the victim group takes it as an even larger sign of disrespect and at some point an intentional slight. Then even if the action ceases to hurt anybody on the object level, or is replaced altogether by things that are thematically similar but not object-level harmful, it has become a slight, and continues to hurt, because connotations are hard to erase. I don’t really know if this happens—I don’t keep up with current offense. More informed opinions welcome.

This theory predicts that actions would often be offensive in spite of probably not directly harming anyone on the object level. I think this does happen. I also guess that it leads to some confusion around whether other people are just pretending to be offended. (I also expect people sometimes are pretending to be offended, because often there are incentives to, and at least some people respond to incentives sometimes).

I also wonder if something like this explains why people jump on random insensitive statements that weakly suggest offensive views, even when there is no chance that the person holds the offensive view in question. If I really believed that someone thought I was good at drawing, but I also heard them accidentally momentarily imply to someone else that my drawings were rubbish, I would figure that they weren’t very interested in whether this might hurt me. Also that they might be trying to intentionally anger me. And people intentionally trying to anger me can be angering.

I posit that offense should almost always be out of proportion to the action that caused it. Hurtful actions automatically snowball into being more hurtful, and the offense of the victim is a response to the hurt accrued by the time the snowball lands.

Sometimes I have had crushes on people, and then all kinds of miscellaneous characteristics they had seemed good. Not just their face or their sense of style or the exact way they pronounce my name. But also things that would usually be considered unattractive. For instance, if they are balding, I might suddenly find myself excited by sparse head stubble, when I had previously liked luxuriant hair. And then subsequently I would be more attracted to every other balding guy I met.

I think this is not just directly because the person having those characteristics makes the characteristics by association the most excellent characteristics a person could have. Though that is maybe part of it (your face reminds me of…you!)

I think it is also because I implicitly infer that the person in question likes those characteristics, and I expect people to like me more if I like the things they like. For instance, if they are grumpy and have crumpled clothes, I think I implicitly infer that they like people being grumpy and wearing crumpled clothes, and that if I favor those things too, it will help us be friends. And I can appreciate a pretty wide range of things, so I implicitly give attention to the ones that are helpful.

So I suppose that I must implicitly believe everyone likes almost all of their characteristics. Explicitly, I think this is unlikely to be true. Though I do expect people relate more to people who share their characteristics, whether or not they like the characteristics. So maybe that is what I’m implicitly going for.

All this leads me to think that that my brain is probably doing a milder version of the thing it does with crushes with respect to other people who I like in less extreme ways all the time. “Ooh—I guess you like being mildly irritated! I can do that too! Grr. Do you like me?” It is just only so strong as to be introspectively perceptible in the case of crushes. Which I guess matches the observation that people copy each other a lot.

I have long had the abstract impression that I should choose who I spend much time with carefully because company makes an alarmingly large difference to one’s own behavior. But the way that my brain updates on crushes makes that concern feel more viscerally real to me. Happily (not coincidentally) current company seems pretty good. Though unusual, so probably I don’t give things like religiosity and being athletic proper thought. These concerns are is not news, but a new angle from which to feel like it is actually a real problem and not just one of those problems that it would be virtuous to be troubled by.

I

It is hard to spread misinformation, because information spreads too, and they eventually run into each other and explode.

If a person wants to lie then, they can be better off to make words correspond to different things for different people, so that even when people hear the information it sounds the same as the misinformation.

For instance, suppose you buy tea in Alphaland and sell it in Betaland. As a dishonest businessperson, you would like it if the people in Alphaland believed tea was cheaper that people in Betaland believed it was. However if there are two different verbal sentences kicking around about the price of tea, they will eventually run into each other, because sentences can spread fast.

A different solution is to corner the market for tea weighing devices in at least one nation. Then make them all give slightly biased readouts. Now tea costs the same amount per pound in the two places, but you just sell more pounds than you buy. The information and the misinformation both sound like “tea costs $10/lb”. Tea measuring devices cross the sea slower than words, so this might be more sustainable.

Relatedly, if you wanted to not have your children informed about Santa Claus, you might just call him something else—e.g. Joulupukki—in your home. If you want, you can tell them there is a probable faker called Santa Clause and it is a matter of controversy whether he is the real deal like Joulupukki. Because words refer to unusual things, the information—‘Santa Clause isn’t real’—sounds just like your misinformation.

This can really only work if people are sufficiently isolated that the differences in meanings don’t become obvious, but that sometimes happens.

II

I’m not much in favor of misinformation. But one time I was young and desperate and I did something like this.

From when I was a young teenager I was substantially in charge of raising my three younger brothers, and (because I was not a good necromancer) I had to keep the violence within certain limits.

First (I think) I tried to be nice. I sat down and talked to them about what had happened, and if someone had clearly been vicious, I sent him to his room or something. But it seemed that only punishing people who are clearly guilty leads to not-very-conscionable levels of violence to innocent children.

I pondered justice and mercy. I construed the situation game theoretically. I experimented with different rules and punishment regimes. The children bit themselves just to spite each other (I think).

I gave up on figuring out guilt, and tried just sending everyone to their room for every fight. They started fights just to see the innocent victim punished. They also destroyed parts of the house and its contents if they were annoyed about being sent to their rooms unjustifiedly, so sending people to their rooms was kind of costly.

I wondered whether children are oblivious to incentives, or just wisely refuse to negotiate with authorities, forcing the authorities to give up. But I couldn’t really give up, because I didn’t have any other options (I was a step ahead in the not negotiating, as the children might have realized if they had read Schelling).

This all took up a lot of the time that I wasn’t at school, if I recall. Every time I would sit down to read a book or something, I would be interrupted by shrieking. I really don’t like shrieking. At this point, I don’t even like the sound of joyful childish laughter, I think because I associate it with joyful childish unapologetic cruelty and hatred. But furthermore, I don’t like being interrupted every five minutes to have a big argument with some children. So I really didn’t like this situation.

My brothers were ‘meant to’ go to bed in the evening. If I started encouraging them to go to bed at about 10pm, that gave the four of us enough time to argue about whether they should or not for three hours before an adult came home and became angry about how the children weren’t in bed. At which point my brothers would go to bed, because the adults were bigger and more exhausted and more authoritative than me.

At some point I realized I had been thinking about things all wrong. All peace really required was for my brothers to believe that it was almost 1am. And my brothers’ beliefs about what time it was were almost entirely dependent on seven or so clocks. And clocks have little dials on the back of them that you can turn around to change where their hands are.

It was somewhat complicated, because there were a bunch of external signs about what time it was. For instance, school would end at 3pm or so. So it had to be 3.30 or so when the children returned home. After that I would gradually change all seven or so visible clocks in the house forward half an hour or an hour at a time, several times through the afternoon and evening. Then by about 8pm it would be past midnight, and the children would hurry off to bed before any adults came home. Then I would get hours (!!!) of solitude (!!!) and peace (!!!)

This was somewhat complicated by the TV schedule, which I said must have been printed for a state in a different time zone with somewhat different programming.

It was also complicated by me messing up one time, and my brother noticing that it was still light at midnight. But a conspiracy didn’t occur to him, and he dismissed it as ‘funny weather today’.

Ultimately the entire scheme was short lived, but only ended by my mother and stepfather objecting to it. My brothers didn’t suspect until I told them about it later.

So, that’s another example. I sometimes wonder if there is more of this kind of thing in the world.