Category Archives: 1

Resolving Paradoxes of Intuition

Cross posted from Overcoming Bias. Comments there.

***

Shelly Kagan gave a nice summary of some problems involved in working out whether death is bad for one. I agree with Robin’s response, and have postedbefore about some of the particular issues. Now I’d like to make a more general observation.

First I’ll summarize Kagan’s story. The problems are something like this. It seems like death is pretty bad. Thought experiments suggest that it is bad for the person who dies, not just their friends, and that it is bad even if it is painless. Yet if a person doesn’t exist, how can things be bad for them? Seemingly because they are missing out on good things, rather than because they are suffering anything. But it is hard to say when they bear the cost of missing out, and it seems like things that happen happen at certain times. Or maybe they don’t. But then we’d have to say all the people who don’t exist are missing out, and that would mean a huge tragedy is happening as long as those people go unconceived. We don’t think a huge tragedy is happening, so lets say it isn’t. Also we don’t feel too bad about people not being born earlier, like we do about them dying sooner. How can we distinguish these cases of deprivation from non-existence from the deprivation that happens after death? Not in any satisfactorily non-arbitrary way. So ‘puzzles still remain’.

This follows a pattern common to other philosophical puzzles. Intuitions say X sometimes, and not X other times. But they also claim that one should not care about any of the distinctions that can reasonably be made between the times when they say X is true and the times when they say X is false.

Intuitions say you should save a child dying in front of you. Intuitions say you aren’t obliged to go out of your way to protect a dying child in Africa. Intuitions also say physical proximity, likelihood of being blamed, etc shouldn’t be morally relevant.

Intuitions say you are the same person today as tomorrow. Intuitions say you are not the same person as Napoleon. Intuitions also say that whether you are the same person or not shouldn’t depend on any particular bit of wiring in your head, and that changing a bit of wiring doesn’t make you slightly less you.

Of course not everyone shares all of these intuitions (I don’t). But for those who do, there are problems. These problems can be responded to by trying to think of other distinctions between contexts that do seem intuitively legitimate, reframing an unintuitive conclusion to make it intuitive, or just accepting at least one of the unintuitive conclusions.

The first two solutions – finding more appealing distinctions and framings – seem a lot more popular than the third – biting a bullet. Kagan concludes that ‘puzzles remain’, as if this inconsistency is an apparent mathematical conflict that one can fully expect to eventually see through if we think about it right. And many other people have been working on finding a way to make these intuitions consistent for a while. Yet why expect to find a resolution?

Why not expect this contradiction to be like the one that arises if you claim that you like apples more than pears and also pears more than apples? There is no nuanced way to resolve the issue, except to give up at least one.  You can make up values, but sometimes they are just inconsistent. The same goes for evolved values.

From Kagan’s account of death, it seems likely that our intuitions are just inconsistent. Given natural selection, this is not particularly surprising. It’s no mystery how people could evolve to care about the survival of they and their associates, yet not to care about people who don’t exist. Even if people who don’t exist suffer the same costs from not existing. It’s also not surprising that people would come to believe their care for others is largely about the others’ wellbeing, not their own interests, and so believe that if they don’t care about a tragedy, there isn’t one. There might be some other resolution in the death case, but until we see one, it seems odd to expect one. Especially when we have already looked so hard.

Most likely, if you want a consistent position you will have to bite a bullet. If you are interested in reality, biting a bullet here shouldn’t be a last resort after searching every nook and cranny for a consistent and intuitive position. It is much more likely that humans have inconsistent intuitions about the value of life than that we have so far failed to notice some incredibly important and intuitive distinction in circumstances that drives our different intuitions. Why do people continue to search for intuitive resolutions to such problems? It could be that accepting an unintuitive position is easy, unsophisticated, unappealing to funders and friends, and seems like giving up. Is there something else I’m missing?

Moving blogs

Today Overcoming Bias becomes a group blog again, and I become one of the group. Robin will keep blogging, joined by Robert Wiblin and me. The other two are my good friends, and among my most respected intellectual influences, so it should be fun! We also hope that between us we can better produce regular enough output to make it worth your while visiting, without taking too much time away from other projects that we all have.

I might cross post my posts back here, for the sake of completeness. However it probably won’t be timely, and I might turn off the comments to keep the conversation in one place. So update your bookmarks/RSS/etc!

Do strange scenarios help us ask why not?

People are working on making robot cars communicate, with pedestrians for instance.

Notice that the apparent benefit of having cars communicate with pedestrians doesn’t actually have much to do with robots driving the cars. If having cars signal to pedestrians is useful, probably so is having drivers signal to pedestrians. Yet current cars and driving norms hardly provide for this at all. Many a time I have thought about this when trying to cross a road when there is a car coming toward me that seems to be slowing down, kind of, and whose windscreen I can’t really see through. Is the driver waving to me? Eating a sandwich? Hard to tell, so I won’t take my chances. Ah, now he’s stopped. And he’s annoyed. Or swatting a fly. Does that mean he’s about to go? Hard to tell, maybe I’ll just wait a sec to be sure. Now he’s really annoyed – annoyed enough to give up and drive on?… If only there were some little signal that meant ‘while this signal is on, I see you and am stopping for you’.

This is not my real point, but an example. Thinking about a strange future of robot cars causes us to make predictions and envision potentially valuable additions to it that have little to do with robot cars. Similarly, thinking about future AI development causes people to wonder if sudden leaps in technological capacity could cause a small portion of humanity to get far ahead of the rest, or if human values might be lost in the long run. These issues are not specific to AI. Yet when we look at the world around us we seem less likely to see ways to improve it, or to wonder why no groups of humans do get ahead of the rest technologically, or even notice that technological changes tend to be relatively small, or to ask what is becoming of our values.

In general it seems that thinking about strange scenarios causes people expect things to happen which have little to do with the scenarios. Since they have little to do with the scenarios, it makes sense to ask why they haven’t already happened, or whether we could already benefit from them.

Some men see things as they are and say, why? I dream of things the way they never were and say, why not?

– Robert F. Kennedy, after George Bernard Shaw

Dreaming of the way things never were seems more impressive, difficult, and useful. Perhaps thinking of strange scenarios is one way to do it more easily.

A scene I once saw

(inaccurately recounted)

Ms. Knox: When any of you feels ready, you can move in to the center of the circle, hold the stone, and tell us all about your feelings about what we are doing. Listen to the trees moving, encouraging you.

Sarah: I feel really proud. Young people are so passionate about the environment. Everyone will have to believe in us when they see how much we care.

Amanda: Excited! I feel like we are going to be part of a really positive change, across the world. It’s so great to be here now, when this is happening.

Marie: I’m just really glad to be here with so many likeminded people. When nobody around you sees what’s possible, it can be really disillusioning, but here I feel like everyone cares so much.

Linda: I feel really inspired by what the others are saying!

Becky: I’m so hopeful when I see all this engagement. I believe we can all stay passionate and keep the movement going until we are old, and inspire the new youth!

Odette: Irritated! I have so many things I would enjoy doing more than saving the environment, both this weekend and for the rest of my life. Preventing ecological catastrophe is very important, but I’d obviously much much prefer that someone else had done it already, or that it never needed doing. It’s extremely disappointing that after this many generations nobody’s got around to the most obvious solutions like taxing the big externalities. These things are not even interesting to think about. In a perfect world it would be nice to play video games most of the time, but I’m at least as frustrated that I won’t even get to work on the interesting altruistic endeavors.

***

Why is this so rare?

More convergent instrumental goals

Nick Bostrom follows Steve Omohundro in exploring the types of instrumental goals that intelligences with arbitrary ultimate goals might converge on. This is important for two reasons. First, it means predicting the behavior of arbitrary intelligences might be a tiny bit easier than you’d think. Second, because it draws attention to the difficulty of creating a creature that doesn’t want to get mixed up in taking resources and seeking longevity and that sort of thing.

Between Nick and Steve we have these convergent instrumental goals:

  1. Self-preservation
  2. Preserve your values
  3. self-improvement
  4. rationality
  5. other cognitive enhancement
  6. technological perfection
  7. get resources
  8. Avoid counterfeit utility

I think acquiring information is included in cognitive enhancement here, though to me it seems big and different enough that I’d put it by itself.

I’d like to add three more, incidentally all to do with interacting with other intelligences. So not relevant if you are the one all powerful singleton, but  handy if there are a number of AIs which are of a similar enough level that they might be useful to one another (like sheep and humans are, or humans and humans are). Here’s my list:

Spread your values
This can be achieved by making other entities which have at least some of the same values, or more traditionally by buying advertising space, exercising social influence, or having children. This makes sense for the same reason that ensuring that you maintain your values in the future does. If you think of yourself as just your values (as Bostrom mentions might be appropriate for some kinds of intelligences), spreading values can come under the heading of self-preservation perhaps (or reproduction). But usually we don’t think like this. If I convince you that Bach is pretty good, we don’t think of this as me replacing part of you with part of me. So I think it’s worth having a different category.

Social influence
If you can’t get others to have your values, the next best thing is to get them to do what you want anyway. This can be done in many ways, e.g. deceit, deals, forming relationships with useful people, sending good signals, being likable. Perhaps manipulating others comes under perfection of technology: making friends is in a way like better programming your computer to do what you type in. Both can be seen as investing some effort in arranging a complex system such that in future you will get what you want from it with less input. But not that close, so I give these things a new category. The extent to which it is worth influencing other intelligences depends on how useful they are. If the world only had me and sheep and stupider things in it, influencing sheep would be handy, but not nearly as handy as social skills are to me in this world. The next can be thought of as a subcategory of this one.

Acquire commonly valued goods
Nick and Steve mention acquiring ‘resources’, but from their elaboration it seems like they mean goods that are direct inputs to the intelligence’s goals. An instrumental goal to acquiring directly useful resources is acquiring stuff that is valuable to someone else. This apparently roundabout route is common because of comparative advantage. So while an AI would presumably like to have energy and fast hardware, it would also like a yacht or intellectual property rights for a new kind of frozen dinner, assuming anyone exists who will trade. With few assumptions, acquiring something like the latter will be easier than acquiring the former directly. This is pretty much what humans spend their working lives on.