Tag Archives: human nature

Epistemology of evilness

Most everyone seems to think that a big reason for bad things happening in the world is that some people are bad. Yet I almost never see advice for telling whether you yourself are a bad person, or for what to do about it if you seem to be one. If there are so many bad people, isn’t there a very real risk that you are one of them?

Perhaps the model is one where you automatically know whether you are good or bad, and simply choose which to be. So the only people who are bad are those who want to be bad, and know that they are bad. But then if there is this big population of bad people out there who want to be bad, why is so little of the media devoted to their interests? There’s plenty on how to do all the good things that a good person would want to do, such as voting for the benefit of society, looking after your children, buying gifts, expressing gratitude to friends, holding a respectable dinner, pleasing your partner. Yet so little on scamming the elderly, effectively shaking off useless relatives, lying credibly, making money from investments that others are too squeamish to take, hiding bodies. Are the profit-driven corporate media missing out on a huge opportunity?

If there aren’t a whole lot of knowingly bad people out there who want to be bad, and could use some information and encouragement, then either there aren’t bad people at all, or bad people don’t know that they are bad or don’t want to be bad. The former seems unlikely, by most meanings of ‘bad’. If the latter is true, why are people so blase about the possibility that they themselves might be bad?


Prompted by the excellent book Harry Potter and the Methods of Rationality, in which there is much talk of avoiding becoming ‘dark’, in stark contrast to the world that I’m familiar with. If you enjoy talking about HPMOR, and live close to Pittsburgh, come to the next Pittsburgh Less Wrong Meetup.

When to explain

It is commonly claimed that humans’ explicit conscious faculties arose for explaining to others about themselves and their intentions. Similarly when people talk about designing robots that interact with people, they often mention the usefulness of designing such robots to be able to explain to you why it is they changed your investments or rearranged your kitchen.

Perhaps this is a generally useful principle for internally complex units dealing with each other: have some part that keeps an overview of what’s going on inside and can discuss it with others.

If so, the same seems like it should be true of companies. However my experience with companies is that they are often designed specifically to prevent you from being able to get any explanations out of them. Anyone who actually makes decisions regarding you seems to be guarded by layers of people who can’t be held accountable for anything. They can sweetly lament your frustrations, agree that the policies seem unreasonable, sincerely wish you a nice day, and most importantly, have nothing to do with the policies in question and so can’t be expected to justify them or change them based on any arguments or threats you might make.

I wondered why this strategy should be different for companies, and a friend pointed out that companies do often make an effort at more high level explanations of what they are doing, though not necessarily accurate: vision statements, advertisements etc. PR is often the metaphor for how the conscious mind works after all.

So it seems the company strategy is more complex: general explanations coupled with avoidance of being required to make more detailed ones of specific cases and policies. So, is this strategy generally useful? Is it how humans behave? Is it how successful robots will behave?*

Inspired by an interaction with ETS, evidenced lately by PNC and Verizon

*assuming there is more than one

Romance is magical

People seem to generally believe they have high romantic standards, and that they aren’t strongly influenced by things like looks, status and money. Research says our standards aren’t that high, that they drop if the standard available drops for a single evening, and that superficial factors make more of a difference than we think. Our beliefs about what we want are wrong. It’s not an obscure topic though; the evidence should be in front of us. How do we avoid noticing? We’re pretty good at not noticing things we don’t want to – we can probably do it unaided. Here there is a consistent pattern though.

Consider the hypothesis that there is approximately one man in the world for me. I meet someone who appears to be him within a month of looking. This is not uncommon, though it has a one in many million chance of happening under my hypothesis, if I look insanely hard. This should make me doubt my hypothesis in favor of one where there are several, or many million men in the world for me. What do I really do? Feel that since something so unlikely (under the usual laws of chance) occurred it must be a sign that we were really meant for each other, that the universe is looking out for us, that fate found us deserving, or whatever. Magic is a nice addition to the theory, as it was what we wanted in the relationship anyway. Romantic magic and there being a Mr Right are complimentary beliefs, so meeting someone nice confirms the idea that there was exactly one perfect man in the world rather than suggesting it’s absurd.

I can’t tell how serious anyone is about this, but ubiquitously when people happen to meet the girl of their dreams on a bus where they were the only English speaking people they put it down to fate, rather than radically lowered expectations. When they marry someone from the same small town they say they were put there for each other. When their partner, chosen on grounds of intellectual qualities, happens to also be rich and handsome their friends remark at how fortune has smiled on them. When people hook up with anyone at all they tell everyone around how unlikely it was that they should both have been at that bus stop on that day, and how since somehow they did they think it’s a sign.

We see huge evidence against our hypothesis, invoke magic/friendly-chance as an explanation, then see this as confirmation that the original magic-friendly hypothesis was right.

Does this occur in other forms of delusion? I think so. We often use the semi-supernatural to explain gaps caused by impaired affective forecasting. As far as I remember we overestimate strength of future emotional responses, tend to think whatever happens was the best outcome, and whatever we own is better than what we could have owned (e.g. you like the children you’ve got more than potential ones you could have had if you had done it another day). We explain these with ‘every cloud has a silver lining’, or ‘everything happens for a reason’, or ‘it turns out it was meant to happen – now I’ve realised how wonderful it is to spend more time at home’, ‘I was guided to take that option – see how well it turned out!’ or as happens often to Mother; ‘the universe told me to go into that shop today, and uncannily enough, there was a sale there and I found this absolutely wonderful pair of pants!’.

Supernatural explanations aren’t just for gaps in our understanding. They are also for gaps between what we want to believe and are forced by proximity to almost notice.

The origins of virtue

I read Matt Ridley’s ‘The origins of virtue’ just now. It was full of engaging anecdotes and irrelevant details, which I don’t find that useful for understanding, so I wrote down the interesting points. On the off chance anyone else would like a summary, I publish it here. I recommend reading it properly. Things written in [here] are my comments.



The aim of this book: How did all this cooperation and niceness, especially amongst humans, come about evolutionarily?

Chapter 1

There are benefits to cooperation: can do many things at once, [can avoid costs of conflict, can enjoy other prisoners’ dilemmas, can be safer in groups]

Cooperation occurs on many levels: allegiances, social groups, organisms, cells, organelles, chromosomes, genomes, genes.

Selfish genes explain everything.

Which means it’s possible for humans to be unselfish.

There are ubiquitous conflicts of interest to be controlled in coalitions at every level.


Relatedness explains most groupishness ( = like selfishness, but pro-group). e.g. ants, naked mole rats.

Humans distribute reproduction, so aren’t closely related to their societies. They try to suppress nepotism even. So why all the cooperation?

Division of labour has huge benefits (trade isn’t zero sum)

[cells are cool because they have the same genes, so don’t mutiny, but different characters so benefit from division of labour]

Division of labor is greater in larger groups, and with better transport.

There is a trade-off between division of labour and benefits of competition.

By specialising at individual level a group can generalise at group level: efficiently exploit many niches.

Division of labour between males and females is huge and old.


Prisoners’ dilemmas are ubiquitous.

Evolutionarily stable strategies = nash equilibria found by evolution.

Tit-for-tat and related strategies are good in iterated prisoners’ dilemmas.

This is because they are nice, retaliatory, forgiving, and clear.

If a combination of strategies play against one other repeatedly, increasing in number according to payoffs, the always-defectors thrive as they beat the always-cooperators, then the tit-for-taters take over as the defectors kill each other.

Reciprocity is ubiquitous in our society.

Hypothesis: it’s an evolutionarily stable strategy. It allowed us to benefit from cooperation without being related. This has been a major win for our species.

Reciprocity isn’t as prevalent between related individuals (in ours and other species).

Tit-for-tat can lead to endless revenge :(


Reciprocity requires remembering many other individuals and their previous behavior. This requires a large brain.

Reciprocity requires meeting the same people continually. Which is why people are nastier in big anonymous places.

Other strategies beat tit-for-tat once tit-for-tat has removed nastier strategies. Best of these is pavlov, or win-stay/lose-shift, especially with learned probabilities.

In asynchronous games ‘firm-but-fair’ is better – similar to pavlov, but cooperates [once presumably] after being defected against as a cooperator in the last round.

In larger populations reciprocity should be less beneficial – most interactions are with those you won’t see again.

Boyd’s suggestion: this is the reason for morality behaviour, or punishing those who don’t punish defection.

Another solution: social ostracism: make choosing who to play with an option.

A strategy available to humans is prediction of cooperativeness in advance. [Why can we do this? Why don’t we evolve to not demonstrate our lack of cooperativeness? Because others evolve to show their cooperativeness if they have it? There are behaviours that only make sense if you intend do be cooperative.]


We share food socially a lot, with strangers and friends. Not so much other possessions. Sex is private and coveted.

Meat is especially important in shared meals.

Hypothesis: meat hunting is where division of labour was first manifested.

Monkey males share meat with females to get sex, consequently hunting meat more than would be worth it for such small successes otherwise.

Hypothesis: humans do this too (some evidence that promiscuous natives hunt more), and the habit evolved into a sexual division of labour amongst married couples (long term relationships are usual in our species, but not in chimps). Males then benefit from division of labour, and also feeding their children.

Hypothesis: sexual division of labour fundamental to our early success as a species – neither hunting or gathering would have done alone, but together with cooking it worked.

Hypotheses: food sharing amongst non-relatives could have descended from when males of a tribe were mostly related, or from the more recent division of labour in couples.

Chimps share and show reciprocity behaviour, but do not offer food voluntarily [doesn’t that suggest that in humans its not a result of marriage related sexual division?]

Why do hunter-gatherers share meat more, and share more on trips?

Hypotheses: 1. meat is cooperatively caught, so have to share to continue cooperation. 2. High variance in meat catching – sharing gives stable supply.

What stops free-riding then?


Mammoth hunting introduced humans to significant public goods. You can’t not share a mammoth, especially if others have spear throwers. [mammoth hunting should have started then when it became easier to kill a mammoth than to successfully threaten to kill a tribesman who killed a mammoth]

Tolerated theft: the idea that people must share things where they can’t use all of them, and to prevent others from taking parts is an effort. That is, TT is what happens once you’ve caught a public good (e.g. mammoth). Evidence that this isn’t what happens in reality; division seems to be controlled. Probably reciprocity of some sort (argument over whether this is in the form of material goods or prestige and sex). Evidence against this too; idle men are allowed to share (if the trade is in sex, they aren’t the ones the trading is aimed at, and miss out on the sex trade).

Alternative hypothesis: is treated as a public good, but so big that it’s possible to sneak the best bits to girls and get sex.

Trade across time (e.g. in large game) reduces exposure to fluctuations in meat.

Hypothesis: hunter-gatherers are relatively idle because they have to share what they get, so stop getting things after their needs are fulfilled.

Hypotheses: when people hoard money they are punished by their neighbours because they are defecting in the reciprocal sharing that usually takes place, yet they have no incentive to share if they have an improbably large windfall – the returned favours won’t be as good. Alternatively can be seen as tolerated theft: punishment for not sharing is an attempt to steal from huge good.

When instincts for reciprocity are in place, gifts can be given ‘as weapons’. That is, to force future generosity from the recipient.

Gift giving is less reciprocal (still prevalent, just not carefully equal) amongst human families than amongst human allies.

Gifts can then also signal status; ostentatious generosity demands reciprocity – those who can’t lose face. The relative benefit of buying status this way depends on the goods – perishables may as well be grandly wasted. In this case reciprocity is zero sum: no benefits from division of labour, status cycle is zero sum.


Humans are better at solving Wason test when it is in terms of noticing cheating than in terms of other social contexts, or abstract terms.

Hypothesis: humans have an ‘exchange organ’ in their brains, which deals with calculating related to social contracts. This is unique amongst animals. Evidence: brain damage victims and patients who fail all other tests of intelligence except these, anthropomorphic attitudes to nature heavily involve exchange, anthropomorphizing of objects heavily involves exchange related social emotions (anger, praise).

Moral sentiments appear irrational, but overcome short term personal interests for long term genetic gains.

Commitment problem: when at least one side in a game has no credible threat if the other defects, how can cooperation occur? The other can’t prove they will commit. e.g. kidnap victim can’t prove she won’t go to police, so kidnapper must kill her even if both would be better off if he let her go in return for her silence.

Various games have bad equilibria for rational players in one off situations, but emotions can change things. e.g. prisoners’ dilemma is solved if players have guilt and shame. Where player would be irrational to punish other for defection (punishment costly to implement, loss already occurred), anger produces credible threat (will punish in spite of self).

Many emotions serve to alter the rewards of commitment problems, by bringing forward costs and benefits.

For this to work, emotions have to be hard to fake. Shouldn’t defectors who are good at faking emotions invade a population of people who can’t? No, because in the long run the people who can’t find each other and cooperate together. [that’s what would happen anyway – you would cooperate the first time, then don’t go back if the other defects. Commitment should be a problem largely in one off games – are more emotions shown in those things? In one off games can’t have the long run to find people and make good liars pay].

Emotions make interests common, which stops prisoners’ dilemmas. Interests of genes are not common, so emotions must be shared with other emotional ones.

Ultimatum game variations suggest that people are motivated more by reciprocity than by absolute fairness.

People lacking social emotions due to brain damage are paralyzed by indecision as they try to rationally weigh information.

We like and praise altruism much more than we practice it. Others’ altruism and our looking altruistic are useful, whereas our own selfishness is. [why aren’t people who behave like this invaded by slightly more altruistic ones who don’t cooperate with them? Why is the equilibrium at being exactly as selfish as we are? Signaling means that everyone looks more altruistic than they are, so everyone is less altruistic than they would be if others were maximally altruistic?]

Hypothesis: economics and evolutionary biology are held in distrust because talking about them doesn’t signal belief in altruism etc. Claiming that people or genes are selfish suggests that you are selfish.


Cooperation began (or is used primarily in monkey society) in competition and aggression.

The same ‘tricks’ will be discovered by evolution as by thought [if their different aims don’t matter], so if we share a behaviour with animals it’s not obvious that it’s evolved in our case, though often it is.

Our ancestors were: social, hierarchical (especially amongst males), more egalitarian and with less rigid hierarchies than monkeys.

Differences between primates:

Monkey hierarchies rely on physical strength more than chimp ones, which rely on social manipulation.

Baboons use cooperation to steal females from higher ranking males, chimps use it to change the social hierarchy.

Chimp coalitions are reciprocal, unlike monkeys.

Power and sexual success are had by coalitions of weaker individuals in chimps and humans.

Bottlenose dolphins (the only species other than us with brain:body ratio bigger than chimps): males have coalitions of 2-3 which they use to kidnap females. All mate with her. These coalitions join to form super-coalitions to steal females from other coalitions. This is reciprocal (on winning, one coalition will leave the female with the other coalition in the super-coalition, in return for the favor next time)

Second order alliances seem unique to dolphins and humans.

Chimp males stay in a troop while females leave, with monkeys it is the other way around. Could be related to aggressive xenophobia of chimp males. Seems so in human societies: matrilineal societies are less fighty.

Chimp groups, rather than individuals, possess territory (rare, but not unique: e.g. wolves).

Hypothesis: this is an extension of the coalition building that occurs for gaining power in a group. Alpha males prevent conflict within the group, making it stable, which is good for all as they are safer from other groups if they stick together.

Humans pursue status through fighting between groups, whereas chimps only do it within groups [how do you know?]


Group selection can almost never happen.

Large groups cooperating are often being directly selfish (safer in shoal than alone).

50:50 sex ratio is because individual selection stronger than group selection. A group would do better by having far more females, yet a gene to produce males would make you replicate much faster, bringing ratio back.

Humans appear to be exception: culturally, not genetically, different groups compete.

Conformism would allow group characteristics to persist long enough that there would be group selection before groups dissolved or were invaded by others’ ideas.

Why would conformism evolve?

Hypothesis: we have many niches which require different behaviors. If you move it’s beneficial to copy your behavior from your neighbors.

Imitation should be more beneficial if there are more people doing it; better to copy something tried by many than the behavior of one other. How did it get started then?

Hypothesis: seeing what is popular amongst many gives you information.

Hypothesis: keeps groups together. If receptive to indoctrination about altruism we will find ourselves in more successful groups [I don’t follow this].

Humans don’t actually live in groups; they just perceive everything in terms of them.

A persons’ fate isn’t tied to that of their group. They don’t put the group’s wellbeing first. They are groupish out of selfishness – it’s not group selection.

Ritual is universal, but details of it are particular.

Hypothesis: Ritual is a means to conformity keeps groups together in conflict, and they survive [How would this begin? Why ritual? Why do they have to be different? Why is conformity necessary to keep groups together? Seems just that we are used to conformity being linked to staying together we assume one leads to the other].

Music and religious belief seem to have similarly group grouping properties.

Cooperation within groups seems linked to xenophobia outside them [cooperation for safety in conflict is of course. What about cooperation for trade? Has that given us non-xenophobia induced cooperative feelings? Earlier chapters seemed to imply so].


Weak evidence of trade 200,000 years ago – not clear when it started.

Trade between groups is unique to humans.

Trade is the glue of alliances between groups; it appears that some trade is just an excuse for this.

Trading rules predate governments. Governments nationalize preexisting trading systems. e.g. 11th C Europe merchant courts [is this a general trend? why is everything in anecdotes? aargh].

Speculation isn’t beneficial because there is no division of labour [?].


Natives are not ecologically nice. They do not conserve game. They sent many species extinct.

We tie environmentalism up with other morality [is it pro-social morality, as the book has been about, or purity?].

As with other morality, we are more programmed to preach than to practice.

It doesn’t look like people have an instinctive environmental ethic [it’s a big prisoners’ dilemma – can’t we make use of something else in our repertoire?].


Property rights emerge unaided where it is possible to defend them [if you see a tragedy of the commons coming, best to draw up property rights – no reason you will be the free rider].

Nationalization often turns property-divided ‘commons’ into a free for all, as the govt can’t defend it and nobody has reason to protect what they are stealing from.

Ordered and successful systems can emerge without design. e.g. Bali subak traditions could have resulted from all copying any neighbour who did better than them.

Lab experiment suggests that communication encourages a lot of cooperation in tragedy of commons games (better than ability to fine defectors)

If humans can arrange property rights unaided, why all the extinctions last chapter?

Hypothesis: property rights can’t be enforced on moving things. Animals that could have property rights asserted on them did have in some cases. e.g. Beavers.

Hoarding taboo (as a result of reciprocity instinct) is to blame for environmentalist dislike of privatisation as a solution.

Hoarding isn’t allowed in primitive tribes, but as soon as more reliable lifestyle allows powerful individuals to do better by hoarding than relying on social insurance, they do. Yet we retain an aversion to it.


Humans are born wanting to cooperate, discriminate trustworthiness, commit to trustworthiness, gain a reputation, exchange goods and info, and divide labour.

There was morality before the church, trade before the state, exchange before money, social contracts before Hobbes, welfare before rights, culture before Babylon, society before Greece, self interest before Adam Smith and greed before capitalism.

Also tendency to xenophobic groups is well inbuilt.

How can we make use of our instincts in designing institutions?

Trust is fundamental to cooperative parts of human nature being used.

This has been part of an endless argument about the perfectability of man, famously between Hobbes and Rousseau. Also about how malleable human nature is. [The book goes into detail about the argument over the centuries, but it’s an irrelevant story to me].

To say that humans are selfish, especially that their virtue is selfish, is unpopular because saying so encourages it supposedly.

Big state doesn’t make bargains with the individual, engendering responsibility, reciprocity, duty, pride – it uses authority. How do people respond to authority?

Welfare state replaces existing community institutions based on reciprocity and encouraging useful feelings, having built up trust over the years. Centralised replacements like the National Health Service. Mandatory donation → reluctance, resentment. Client feelings changed from gratitude to apathy, anger, drive to exploit the system.

:. Government makes people more selfish, not less.

We must encourage material and social exchange between people, for that is what breeds trust, and trust is what breeds virtue.

Our institutions are largely upshots of human nature, not serious attempts to make the best of it.

Don’t change your mind, just change your brain

The best way to dull hearts and win minds is with a scalpel.

Give up your outdated faith in the pen over the sword! With medical training and a sufficiently sharp but manoeuvrable object of your choice, you can change anyone’s mind on the most contentious of moral questions. All you need to make someone utilitarian is a nick to the Ventromedial Pre­frontal Cortex (VMPC), a part of the brain related to emotion.

When pondering whether you should kill an innocent child to save twenty strangers, eat your pets when they die, or approve of infertile siblings making love in private if they like, utilitar­ians are the people who say “do whatever, so long as the outcome maximises overall happiness.” Others think outcomes aren’t everything; some actions are just wrong. According to research, people with VMPC damage are far more likely to make utilitar­ian choices.

It turns out most people have conflicting urges: to act for the greater good or to obey rules they feel strongly about. This is the result of our brains being composed of interacting parts with different functions. The VMPC processes emotion, so in normal people it’s thought to compete with the parts of the brain that engage in moral rea­soning and see the greatest good for the greatest number as ideal. If the VMPC is damaged, the ra­tional, calculating sections are left unimpeded to dispassionate­ly assess the most compassionate course of action.

This presents practical oppor­tunities. We can never bring the world in line with our moral ide­als while we all have conflicting ones. The best way to get us all on the same moral page is to make everyone utilitarian. It is surely easier to sever the touchy feely moral centres of people’s brains than to teach them the value of utilitarianism. Also it will be for the common good; once we are all utilitarian we will act with everyone’s net benefit more in mind. Partial lo­botomies for the moralistic are probably much cheaper than policing all the behaviours such people tend to disapprove of.

You may think this still doesn’t make it a good thing. The real beauty is that after the procedure you would be fine with it. If we went the other way, everyone would end up saying ‘you shouldn’t alter other people’s brains, even if it does solve the world’s problems. It’s naughty and unnatural. Hmph.’

Unfortunately, VMPC dam­age also seems to dampen social emotions such as guilt and com­passion. The surgery makes utili­tarian reasoning easier, but so too complete immorality, mean­ing it might not be the answer for everyone just yet.

Some think the most impor­tant implications of the research are actually those for moral phi­losophy. The researchers suggest it shows humans are unfit to make utilitarian judgements. You don’t need to be a brain surgeon to figure that out though. Count the number of dollars you spend on unnecessary amusements each year in full knowledge peo­ple starving due to poverty.

In the past we could tell moral questions were prompting action in emotional parts of the brain, but it wasn’t clear whether the activity was influencing the deci­sion or just the result of it. If the latter, VMPC damage shouldn’t have changed actions. It does – so while non-utilitarianism is a fine theoretical position, it is seemingly practiced for egoistic reasons.

Can this insight into cognition settle the centuries of philosophical debate and show utilitarianism is a bad position? No. Why base your actions on what you feel like doing, dis­counting all other outcomes? All it says about utilitarianism is that it doesn’t come easily to the hu­man mind.

This research is just another bit of evidence that moral reasoning is guided by evolution and brain design, not some transcendental truth in the sky. It may still be useful of course, like other skills our mind provides us with, like a capacity to value things, a prefer­ence for being alive, and the abil­ity to tell pleasure from pain.

Next time you are in a mor­ally fraught argument, consider what Ghandi said: “Victory at­tained by violence is tantamount to a defeat, for it is momentary’” He’s right; genetic modification would be more long-lasting. Un­til this is available though, why not try something persuasive like a scalpel to the forehead?

Originally published in Woroni