Tag Archives: values

More convergent instrumental goals

Posted on May 28, 2012 | 5 comments

Nick Bostrom follows Steve Omohundro in exploring the types of instrumental goals that intelligences with arbitrary ultimate goals might converge on. This is important for two reasons. First, it means predicting the behavior of arbitrary intelligences might be a tiny bit easier than you’d think. Second, because it draws attention to the difficulty of creating a creature that doesn’t want to get mixed up in taking resources and seeking longevity and that sort of thing.

Between Nick and Steve we have these convergent instrumental goals:

  1. Self-preservation
  2. Preserve your values
  3. self-improvement
  4. rationality
  5. other cognitive enhancement
  6. technological perfection
  7. get resources
  8. Avoid counterfeit utility

I think acquiring information is included in cognitive enhancement here, though to me it seems big and different enough that I’d put it by itself.

I’d like to add three more, incidentally all to do with interacting with other intelligences. So not relevant if you are the one all powerful singleton, but  handy if there are a number of AIs which are of a similar enough level that they might be useful to one another (like sheep and humans are, or humans and humans are). Here’s my list:

Spread your values
This can be achieved by making other entities which have at least some of the same values, or more traditionally by buying advertising space, exercising social influence, or having children. This makes sense for the same reason that ensuring that you maintain your values in the future does. If you think of yourself as just your values (as Bostrom mentions might be appropriate for some kinds of intelligences), spreading values can come under the heading of self-preservation perhaps (or reproduction). But usually we don’t think like this. If I convince you that Bach is pretty good, we don’t think of this as me replacing part of you with part of me. So I think it’s worth having a different category.

Social influence
If you can’t get others to have your values, the next best thing is to get them to do what you want anyway. This can be done in many ways, e.g. deceit, deals, forming relationships with useful people, sending good signals, being likable. Perhaps manipulating others comes under perfection of technology: making friends is in a way like better programming your computer to do what you type in. Both can be seen as investing some effort in arranging a complex system such that in future you will get what you want from it with less input. But not that close, so I give these things a new category. The extent to which it is worth influencing other intelligences depends on how useful they are. If the world only had me and sheep and stupider things in it, influencing sheep would be handy, but not nearly as handy as social skills are to me in this world. The next can be thought of as a subcategory of this one.

Acquire commonly valued goods
Nick and Steve mention acquiring ‘resources’, but from their elaboration it seems like they mean goods that are direct inputs to the intelligence’s goals. An instrumental goal to acquiring directly useful resources is acquiring stuff that is valuable to someone else. This apparently roundabout route is common because of comparative advantage. So while an AI would presumably like to have energy and fast hardware, it would also like a yacht or intellectual property rights for a new kind of frozen dinner, assuming anyone exists who will trade. With few assumptions, acquiring something like the latter will be easier than acquiring the former directly. This is pretty much what humans spend their working lives on.

5 Comments

Posted in 1

Tagged ,

The appeal of fictional conflict

Posted on March 11, 2012 | 9 comments

Robert Wiblin asks why stories celebrate conflict rather than compromise:

As I was watching the film Avatar and the cinemagoers around me were cheering on the Na’vi heroes in their fight against human invaders, I couldn’t help but wonder how many of us would actually want to live alongside such an uncompromising society…it is hardly an isolated case. In our stories we love idealistic heroes to fight for what they believe in against all odds…

We could tell stories of the countless political compromises reached through well-functioning democratic institutions. We could tell the stories of all the terrible wars that never happened because of careful diplomacy. We could tell the story of the merchant who buys low and sells high, leaving everyone they deal with a little better off. These are the everyday tales which make modern society so great to live in. But will any such movie gross a billion dollars in the near future? I suspect not.

Incidentally, the one line I still remember from Avatar:

They’re not going to give up their home — they’re not gonna make a deal. For what? Lite beer and shopping channel? There’s nothing we have that they want.

Nothing at all…oh, except control over the destruction of everything they care about. You’re right, you really have no bargaining power. As Rob elaborates further, the premise of the extreme conflict was so flimsy, one must infer that it was pretty important to have an extreme conflict.

Rob guesses the popularity of such stubborn warring in our stories is to do with what we subconsciously want our tastes to say about us. When we don’t pay the costs of fictional war, we may as well stand up for principles as strongly as possible.

I think he might be roughly right. But why wouldn’t finding good deals and balancing compromises well be ideals we would want to celebrate? When there are no costs to yourself, why aren’t you itching to go all out and celebrate the most extravagant tales of successful trading and extreme sagas of mutually beneficial political compromise?

I think because there is no point in demonstrating that you will compromise. As a default, everyone can be expected to compromise, because it’s the rational thing to do at the time. However it’s often good to look like you won’t easily compromise, so that other people will try to win you over with better deals. Celebrating ruthless adherence to idealistic principles is a way of advertising that you are insane, for the purpose of improving your bargaining position. If you somehow convince me that you’re the kind of person who would die fighting for their magic tree, I’ll probably try to come up with a pretty appealing deal for you before I even bring up my interest in checking out the deposits under any trees you have.

Of course the whole point of being a bloody-minded idealist is lost if you keep it a secret. So you probably won’t do that. Which means just not going out of your way to celebrate uncompromising fights to the death is a credible signal of willingness to compromise.

9 Comments

Posted in 1

Tagged , , ,

Motivation on the margin of saving the world

Posted on February 8, 2012 | 11 comments

Most people feel that they have certain responsibilities in life. If they achieve those they feel good about themselves, and anything they do beyond that to make the world better is an increasingly imperceptible bonus.

Some people with unusual moral positions or preferences feel responsible for making everything in the world as good as they can make it, and feel bad about the gap between what they achieve and what they could.

In both cases people have a kind of baseline that they care especially about. In the first case they are usually so far above it that nothing they do makes much difference to their feelings. In the second case they are often so far below it that nothing they do makes much difference to their feelings.

Games are engaging when you have a decent chance at both winning and losing. Every move you make matters, so you long to make that one more move. 

I expect the same is true of motivating altruistic consequentialists. I’m not sure how to make achievements on the margin more emotionally salient, but perhaps you do?

11 Comments

Posted in 1

Tagged , , , ,

What’s wrong with advertising?

Posted on November 17, 2011 | 38 comments

These two views seem to go together often:

  1. People are consuming too much
  2. The advertising industry makes people want things they wouldn’t otherwise want, worsening the problem

The reasoning behind 1) is usually that consumption requires natural resources, and those resources will run out. It follows from this that less natural-resource intensive consumption is better* i.e. the environmentalist prefers you to spend your money attending a dance or a psychologist than buying new clothes or jet skis, assuming the psychologist and dance organisers don’t spend all their income on clothes and jet skis and such.

How does the advertising industry get people to buy things they wouldn’t otherwise buy? One practice they are commonly accused of is selling dreams, ideals, identities and attitudes along with products. They convince you (at some level) that if you had that champagne your whole life would be that much more classy. So you buy into the dream though you would have walked right past the yellow bubbly liquid.

But doesn’t this just mean they are selling you a less natural-resource-intensive product? The advertisers have packaged the natural-resource intensive drink with a very non-natural-resource intensive thing – classiness – and sold you the two together.

Yes, maybe you have bought a drink you wouldn’t otherwise have bought. But overall this deal seems likely to be a good thing from the environmentalist perspective: it’s hard to just sell pure classiness, but the classy champagne is much less resource intensive per dollar than a similar bottle of unclassy drink, and you were going to spend your dollars on something (effectively – you may have just not earned them, which is equivalent to spending them on leisure).

If the advertiser can manufacture enough classiness for thousands of people with a video camera and some actors, this is probably a more environmentally friendly choice for those after classiness than most of their alternatives, such as ordering stuff in from France. My guess is that in general, buying intangible ideas along with more resource intensive products is better for the environment than the average alternative purchase a given person would make.  There at least seems little reason to think it is worse.

Of course that isn’t the only way advertisers make people want things they wouldn’t otherwise want. Sometimes they manufacture fake intangible things, so that when you get the champagne it doesn’t really make you feel classy. That’s a problem with dishonest people in every industry though. Is there any reason to blame ‘advertisers’ rather than ‘cheats’?

Another thing advertisers do is tell you about things you wouldn’t have thought of wanting otherwise, or remind you of things you had forgotten about. When innovators and entrepreneurs do this we celebrate it. Is there any difference when advertisers do it? Perhaps the problem is that advertisers tend to remind you of resource intensive, material desires more often than they remind you to consume more time with your brother, or to meditate more. This is somewhat at odds with the complaint that they try to sell you dreams and attitudes etc, but perhaps they do a bit of both.

Or perhaps they try to sell you material goods to satisfy longings you would otherwise fulfil non-materially? For instance recommending new clothes where you might otherwise have sought self-confidence through posture or public speaking practice or doing something worthy of respect. Some such effect seems plausible, though I doubt a huge one.

Overall it seems advertisers probably have effects in both directions. It’s not clear to me which is stronger. But insofar as they manage to package up and sell feelings and identities and other intangibles,  those who care for the environment should praise them.

*This is not to suggest that I believe natural resource conservation is particularly important, compared to using human time well for instance.

38 Comments

Posted in 1

Tagged , , , ,

I am anti-awareness and you should be too

Posted on February 28, 2011 | 25 comments

People seem to like raising awareness a lot. One might suspect too much, assuming the purpose is to efficiently solve whatever problem the awareness is being raised about. It’s hard to tell whether it is too much by working out how much is the right amount then checking if it matches what people do. But a feasible heuristic approach is to consider factors that might bias people one way or the other, relative to what is optimal.

Christian Lander at Stuff White People Like suggests some reasons raising awareness should be an inefficiently popular solution to other people’s problems:

This belief [that raising awareness will solve everything] allows them to feel that sweet self-satisfaction without actually having to solve anything or face any difficult challenges…

What makes this even more appealing for white people is that you can raise “awareness” through expensive dinners, parties, marathons, selling t-shirts, fashion shows, concerts, eating at restaurants and bracelets.  In other words, white people just have to keep doing stuff they like, EXCEPT now they can feel better about making a difference…

So to summarize – you get all the benefits of helping (self satisfaction, telling other people) but no need for difficult decisions or the ensuing criticism (how do you criticize awareness?)…

He seems to suspect that people are not trying to solve problems, but I shan’t argue about that here. At least some people think that they are trying to effectively campaign; this post is concerned with biases they might face. Christian  may or may not demonstrate a bias for these people. All things equal, it is better to solve problems in easy, fun, safe ways. However if it is easier to overestimate the effectiveness of easy, fun, safe things,  we probably raise awareness too much. I suspect this is true. I will add three more reasons to expect awareness to be over-raised.

First, people tend to identify with their moral concerns. People identify with moral concerns much more than they do with their personal, practical concerns for instance. Those who think the environment is being removed too fast are proudly environmentalists while those who think the bushes on their property are withering too fast do not bother to advertise themselves with any particular term, even if they spend much more time trying to correct the problem. It’s not part of their identity.

People like others to know about their identities. And raising awareness is perfect for this. Continually incorporating one’s concern about foreign forestry practices into conversations can be awkward, effortful and embarrassing. Raising awareness displays your identity even more prominently, while making this an unintended side effect of costly altruism for the cause rather than purposeful self advertisement.

That raising awareness is driven in part by desire to identify is evidenced by the fact that while ‘preaching to the converted’ is the epitome of verbal uselessness, it is still a favorite activity for those raising awareness, for instance at rallies, dinners and lectures. Wanting to raise awareness to people who are already well aware suggests that the information you hope to transmit is not about the worthiness of the cause. What else new could you be showing them? An obvious answer is that they learn who else is with the cause. Which is some information about the worthiness of the cause, but has other reasons for being presented. Robin Hanson has pointed out that breast cancer awareness campaign strategy relies on everyone already knowing about not just breast cancer but about the campaign. He similarly concluded that the aim is probably to show a political affiliation.

These are some items given away to promote Bre...

Image via Wikipedia

In many cases of identifying with a group to oppose some foe, it is useful for the group if you often declare your identity proudly and commit yourself to the group. If we are too keen to raise awareness about our identites, perhaps we are just used to those cases, and treat breast cancer like any other enemy who might be scared off by assembling a large and loyal army who don’t like it. I don’t know. But for whatever reason, I think our enthusiasm for increased awareness of everything is given a strong push by our enthusiasm for visible identifying with moral causes.

Secondly and relatedly, moral issues arouse a  person’s drive to determine who is good and who is bad, and to blame the bad ones. This urge to judge and blame should  for instance increase the salience of everyone around you eating meat if you are a vegetarian. This is at the expense of giving attention to any of the larger scale features of the world which contribute to how much meat people eat and how good or bad this is for animals. Rather than finding a particularly good way to solve the problem of too many animals suffering, you could easily be sidetracked by fact that your friends are being evil. Raising awareness seems like a pretty good solution if the glaring problem is that everyone around you is committing horrible sins, perhaps inadvertently.

Lastly, raising awareness is specifically designed to be visible, so it is intrinsically especially likely to spread among creatures who copy one another. If I am concerned about climate change, possible actions that will come to mind will be those I have seen others do. I have seen in great detail how people march in the streets or have stalls or stickers or tell their friends. I have little idea how people develop more efficient technologies or orchestrate less publicly visible political influence, or even how they change the insulation in their houses. This doesn’t necessarily mean that there is too much awareness raising; it is less effort to do things you already know how to do, so it is better to do them, all things equal. However too much awareness raising will happen if we don’t account for there being a big selection effect other than effectiveness in which solutions we will know about, and expend a bit more effort finding much more effective solutions accordingly.

So there are my reasons to expect too much awareness is raised. It’s easy and fun, it lets you advertise your identity, it’s the obvious thing to do when you are struck by the badness of those around you, and it is the obvious thing to do full stop. Are there any opposing reasons people would tend to be biased against raising awareness? If not, perhaps I should reconsider stopping telling you about this problem and finding a more effective way to lower awareness instead.

25 Comments

Posted in 1

Tagged , , , , , , , ,