Category Archives: 1

Cross posted from Overcoming Bias. Comments there.

***

I would like to think I wouldn’t have been friends with slave owners, anti-semites or wife-beaters, but then again most of my friends couldn’t give a damn about the suffering of animals, so I guess I would have been. – Robert Wiblin

I expect the same friends would have been any of those things too, given the right place and period of history. The same ‘faults’ appear to be responsible for most old fashioned or foreign moral failings: not believing that anything bad is happening if you don’t feel bad about it, and not feeling bad about anything unless there is a social norm of feeling bad about it.

People here and now are no different in these regards, as far as I can tell. We may think we have better social norms, but the average person has little more reason to believe this than the average person five hundred years ago did. People are perhaps freer here and now to follow their own hearts on many moral issues, but that can’t make much difference to issues where the problem is that people’s hearts don’t automatically register a problem. So even if you aren’t a slave-owner, I claim you are probably using a similar decision procedure to that which would lead you to be one in different circumstances.

Are these really bad ways for most people to behave? Or are they pretty good heuristics for non-ethicists? It would be a huge amount of work for everyone to independently figure out for themselves the answer to every ethical question. What heuristics should people use?

Cross posted from Overcoming Bias. Comments there.

***

People are often interested in robot ethics. I have argued before that this is strange. I offered two potential explanations:

1. Ethics seems deep and human, so it’s engagingly eerie to combine it with heartless AI
2. People vastly misjudge how much ethics contributes to the total value society creates

A more obvious explanation now: people are just more interested in ethics when the subject is far away, for instance in the future. This is the prediction of construal level theory. It says thinking about something far away makes you think more abstractly, and in terms of goals and ideals rather than low level constraints. Ethics is all this.

So a further prediction would be that when we come to use robots a lot, expertise from robot ethicists will be in as little demand as expertise from washing machine ethicists is now.

Some other predictions, to help check this theory:

• Emerging or imagined technologies should arouse ethical feelings more than present technologies do in general
• International trade should prompt more ethical feelings than local trade
• Stories of old should be more moralizing than stories of now
• Historical figures should be seen in a more moral light than present-day celebrities
• Space travel should be discussed in terms of more moral goals than Earth travel.
• Ethical features of obscure cultures should be relatively salient compared to familiar cultures

More? Which of these are actually true?

There is definitely some conflicting evidence, for instance people feel more compelled to help people in front of them than those in Africa (there was an old OB post on this, but I can’t find it). There are also many other reasons the predictions above may be true. Emerging technologies might prompt more ethical concerns because they are potentially more dangerous for instance. The ethical dimension to killing everyone is naturally prominent. Overall construal level theory still seems to me a promising model for variations in ethical concern.

Added: I’m not confident that there is disproportionate interest compared to other topic areas. I seem to have heard about it too much, but this could be a sampling bias.

Sometimes when people hear obvious arguments regarding emotive topics, they just tentatively accept the conclusion instead of defending against it until they find some half satisfactory reason to dismiss it. Eliezer Yudkowsky calls this ‘clicking’, and wants to know what causes it:

My best guess is that clickiness has something to do with failure to compartmentalize – missing, or failing to use, the mental gear that lets human beings believe two contradictory things at the same time. Clicky people would tend to be people who take all of their beliefs at face value.

The Hansonian explanation (not necessarily endorsed by Robin Hanson) would say something about clicky people tending to operate in Near mode.  (Why?)

pjeby remarks (with 96 upvotes),

One of the things that I’ve noticed about this is that most people do not expect to understand things. For most people, the universe is a mysterious place filled with random events beyond their ability to comprehend or control. Think “guessing the teacher’s password”, but not just in school or knowledge, but about everything.

Such people have no problem with the idea of magic, because everything is magic to them, even science….

Hypothesis: people expect reality to make sense roughly in proportion to how personally responsible for manipulating it they feel. If you think of yourself as in charge of strategically doing something, you are eager to understand how doing that thing works, and automatically expect understanding to be possible. If you are driving a car, you insist the streets fit intuitive geometry. If you are engaging in office politics, you feel there must be some reason Gina said that thing.

If you feel like some vague ‘they’ is responsible for most things, and is meant to give you stuff that you have a right to, and that you are meant to be a good person in the mean time, you won’t automatically try to understand things or think of them as understandable. Modeling how things work isn’t something you are ‘meant’ to do, unless you are some kind of scientist. If you do dabble in that kind of thing, you enjoy the pretty ideas rather than feel any desperate urge for them to be sound or complete. Other people are meant to look after those things.

A usual observation is that understanding things properly allows you to manipulate them. I posit that thinking of them as something you might manipulate automatically makes you understand them better. This isn’t particularly new either. It’s related to ‘learned blankness‘, and searching vs. chasing, and near mode vs. far mode. The followup point is that chasing the one correct model of reality, which has to make sense, straight-forwardly leads to ‘clicking’ when you hear a sensible argument.

According to this hypothesis, the people who feel most personally responsible for everything a la Methods Harry Potter would also be the people who are most notice whether things make sense. The people who less trust doctors and churches to look after them on the way to their afterlives are the ones who notice that cryonics makes sense.

To see something as manipulable is to see it in the same light that science does, rather than as wallpaper. This is expensive, not just because a detailed model is costly to entertain, but because it interferes with saying socially advantageous things about the wallpaper. So you quite sensibly only do it when you actually want to manipulate a thing and feel potentially empowered to do so, i.e. when you hold yourself responsible for it.

Cross posted from Overcoming Bias. Comments there.

***

Warning: this post is technical.

Suppose you know that there are a certain number of planets, N. You are unsure about the truth of a statement Q. If Q is true, you put a high probability on life forming on a given arbitrary planet. If Q is false, you put a low probability on this. You have a prior probability for Q. So far you have not taken into account your observation that the planet you are on has life. How do you update on this evidence, to get a posterior probability for Q? Since you don’t know which is ‘this’ planet, with respect to the model, you can’t update directly on ‘there is life on this planet’, by excluding worlds where this planet doesn’t have life. And you can’t necessarily treat ‘this’ as an arbitrary planet, since you wouldn’t have seen it if it didn’t have life.

I have an ongoing disagreement with an associate who suggests that you should take ‘this planet has life’ into account by conditioning on ‘there exists a planet with life’. That is,

P(Q|there is life on this planet) = P(Q|there exists a planet with life).

Here I shall explain my disagreement.

Nick Bostrom argues persuasively that much science would be impossible if we treated ‘I observe X’ as ‘someone observes X’. This is basically because in a big world of scientists making measurements, at some point somebody will make most mistaken measurements. So if all you know when you measure the temperature of a solution to be 15 degrees is that you are not in a world where nobody ever measures its temperature to be 15 degrees, this doesn’t tell you much about the temperature.

You can add other apparently irrelevant observations you make at the same time – e.g. that the table is blue chipboard – in order to make your total observations less likely to arise once in a given world (at its limit, this is the suggestion of FNC). However it seems implausible that you should make different inferences from taking a measurement when you can also see a detailed but irrelevant picture at the same time than those you make with limited sensory input. Also the same problem re-emerges if the universe is supposed to be larger. Given that the universe is thought to be very, very large, this is a problem. Not to mention, it seems implausible that the size of the universe should greatly affect probabilistic judgements made about entities which are close to independent from most of the universe.

So I think Bostrom’s case is good. However I’m not completely comfortable arguing from the acceptability of something that we do (science) back to the truth of the principles that justify it. So I’d like to make another case against taking ‘this planet has life’ as equivalent evidence to ‘there exists a planet with life’.

Evidence is what excludes possibilities. Seeing the sun shining is evidence against rain, because it excludes the possible worlds where the sky is grey, which include most of those where it is raining. Seeing a picture of the sun shining is not much evidence against rain, because it excludes worlds where you don’t see such a picture, which are about as likely to be rainy or sunny as those that remain are.

Receiving the evidence ‘there exists a planet with life’ means excluding all worlds where all planets are lifeless, and not excluding any other worlds. At first glance, this must be different from ‘this planet has life’. Take any possible world where some other planet has life, and this planet has no life. ‘There exists a planet with life’ doesn’t exclude that world, while ‘this planet has life’ does. Therefore they are different evidence.

At this point however, note that the planets in the model have no distinguishing characteristics. How do we even decide which planet is ‘this planet’ in another possible world? There needs to be some kind of mapping between planets in each world, saying which planet in world A corresponds to which planet in world B, etc. As far as I can tell, any mapping will do, as long as a given planet in one possible world maps to at most one planet in another possible world. This mapping is basically a definition choice.

So suppose we use a mapping where in every possible world where at least one planet has life, ‘this planet’ corresponds to one of the planets that has life. See the below image.

Squares are possible worlds, each with two planets. Pink planets have life, blue do not. Define ‘this planet’ as the circled one in each case. Learning that there is life on this planet is equal to learning that there is life on some planet.

Now learning that there exists a planet with life is the same as learning that this planet has life. Both exclude the far righthand possible world, and none of the other possible worlds. What’s more, since we can change the probability distribution we end up with, just by redefining which planets are ‘the same planet’ across worlds, indexical evidence such as ‘this planet has life’ must be horseshit.

Actually the last paragraph was false. If in every possible world which contains life, you pick one of the planets with life to be ‘this planet’, you can no longer know whether you are in ‘this planet’. From your observations alone, you could be on the other planet, which only has life when both planets do. The one that is not circled in each of the above worlds. Whichever planet you are on, you know that there exists a planet with life. But because there’s some probability of you being on the planet which only rarely has life, you have more information than that. Redefining which planet was which didn’t change that.

Perhaps a different definition of ‘this planet’ would get what my associate wants? The problem with the last was that it no longer necessarily included the planet we are on. So what about we define ‘this planet’ to be the one you are on, plus a life-containing planet in all of the other possible worlds that contain at least one life-containing planet. A strange, half-indexical definition, but why not? One thing remains to be specified – which is ‘this’ planet when you don’t exist? Let’s say it is chosen randomly.

Now is learning that ‘this planet’ has life any different from learning that some planet has life? Yes. Now again there are cases where some planet has life, but it’s not the one you are on. This is because the definition only picks out planets with life across other possible worlds, not this one. In this one, ‘this planet’ refers to the one you are on. If you don’t exist, this planet may not have life. Even if there are other planets that do. So again, ‘this planet has life’ gives more information than ‘there exists a planet with life’.

You either have to accept that someone else might exist when you do not, or you have to define ‘yourself’ as something that always exists, in which case you no longer know whether you are ‘yourself’. Either way, changing definitions doesn’t change the evidence. Observing that you are alive tells you more than learning that ‘someone is alive’.