The debate over dopamine's role in reward: the case for incentive salience and ANGELINA JOLIE!

(The article below by Dr. Kent Berridge on incentive motivation is one of the most important papers I have read on behavioral neuroscience. Nonetheless, when compared to other masters of motivational literature such as Tony Robbins and Zig Ziglar, one feels it needs an extra 'something' to keep your attention on the printed word and of course remind you of the magic of dopamine as you navigate through prose that is admittedly not as exciting as the 4th grade level writing style of  our two masters.) 

The debate over dopamine’s role in reward: the case for incentive salience AND ANGELINA JOLIE!

Kent C. Berridge 

  

Received: 21 March 2006 /Accepted: 20 August 2006 / Published online: 27 October 2006

# Springer-Verlag 2006/ unauthorized Angelina Jolie revision 29 Feb 2012

Abstract

Introduction Debate continues over the precise causal contribution made by mesolimbic dopamine systems to reward. There are three competing explanatory categories: ‘liking’, learning, and ‘wanting’. Does dopamine mostly mediate the hedonic impact of reward (‘liking’)? Does it instead mediate learned predictions of future reward, prediction error teaching signals and stamp in associative links (learning)? Or does dopamine motivate the pursuit of rewards by attributing incentive salience to reward-related stimuli (‘wanting’)? Each hypothesis is evaluated here, and it is suggested that the incentive salience or ‘wanting’ hypothesis of dopamine function may be consistent with  more evidence than either learning or ‘liking’. In brief, recent evidence indicates that dopamine is neither necessary nor sufficient to mediate changes in hedonic ‘liking’ for sensory pleasures. Other recent evidence indicates that dopamine is not needed for new learning, and not sufficient to directly mediate learning by causing teaching or prediction signals. By contrast, growing evidence indicates that dopamine does contribute causally to incentive salience. Dopamine appears necessary for normal ‘wanting’, and dopamine activation can be sufficient to enhance cue-triggered incentive salience. Drugs of abuse that promote dopamine signals short circuit and sensitize dynamic mesolimbic mechanisms that evolved to attribute incentive salience to rewards. Such drugs interact with incentive salience integrations of Pavlovian associative information with physiological state signals. That interaction sets the stage to cause compulsive ‘wanting’ in addiction, but also provides opportunities for experiments to disentangle ‘wanting’, ‘liking’, and learning hypotheses. Results from studies that exploited those opportunities are described here.

Conclusion In short, dopamine’s contribution appears to be chiefly to cause ‘wanting’ for hedonic rewards, more than ‘liking’ or learning for those rewards.

Keywords Accumbens. Reward . Opioid . Dopamine, Basal forebrain . Aversion . Associative learning . Appetite Addiction, ANGELINA JOLIE!

Introduction

Some questions endure for ages, faced by generation after generation. Neuroscientists hope the question, ‘What does dopamine do for reward?’ will not be among them, but it still prompts debate after several decades. Fortunately, the answers to the dopamine question are becoming better. A formal debate on dopamine’s role in reward was held at a Gordon conference on catecholamines in 2005. This article describes the incentive salience case presented in that debate, and compares it to other hypotheses. A debate stance can sometimes help clarify alternative views, and that is the hope here. Therefore, this article is not an exhaustive review of dopamine function. My goal is to provide a useful viewpoint and a critical evaluation oalternatives and to point to new evidence that seems crucial to any decision about what dopamine does for reward.

Dopamine’s causal role in reward

What does dopamine do in reward? This is in essence a question about causation. It asks what causal contribution is made by increases or decreases in dopamine neurotransmission to produce changes in reward-related psychology and behavior. In this article, our focus is on cause and consequence. How to assign causal status to brain events is a complicated issue, but it is not too much an oversimplification to suggest that in practice, the causal question of dopamine’s role in reward has been approached in several experimental ways. One approach is to ask ‘What specific reward function is lost?’ when dopamine neurotransmission is suppressed (e.g., by antagonist drugs, neurotoxin, or other lesions or genetic manipulations that reduce dopamine neurotransmission). That approach asks about dopamine’s role as a necessary cause for reward. It identifies what reward functions cannot be carried on without it. A different approach is to ask ‘What reward function is enhanced?’ by elevations in dopamine signaling (e.g., elevated by agonist drugs, brain stimulation, or hyperdopaminergic genetic mutation). That approach asks about dopamine’s role as a sufficient cause for reward. It asks what reward function a dopamine increase is able to enhance (when other conditions in the brain do not simultaneously change so much as to invalidate hopes of obtaining a specific answer). A third approach is to ask ‘What reward functions are coded?’ by the dopamine neural activations during reward events (e.g., by recording firing of dopamine or related limbic neurons, measuring extracellular dopamine release, or neuroimaging activation in target structures). This question asks about neural coding of function via correlation, often in the hope of inferring causation on the basis of observing correlated functions. Dopamine function is a multifaceted target, so it helps to combine these multiple approaches. What does it contribute to reward? Let’s put on the table the best answers that have survived until today and evaluate each hypothesis for dopamine’s role against the others. These include activation-sensorimotor hypotheses of effort, arousal and response vigor; the hedonia hypothesis of reward pleasure; reward learning hypotheses of associative stamping-in, teaching signals and prediction errors; and the incentive salience hypothesis of reward ‘wanting’. I will describe each of these hypotheses in turn. Then recent experiments that pit hedonia, reward learning, and incentive salience hypothesis against each other will be considered. Their results indicates that dopamine may more directly mediate reward ‘wanting’ than either ‘liking’ or learning about the same rewards.

 Activation-sensorimotor hypothesis

Activation-sensorimotor hypotheses posit dopamine to mediate general functions of action generation, effort, movement, and general arousal or behavioral activation (Dommett et al. 2005; Horvitz 2002; Robbins and Everitt 1982; Salamone et al. 1994; Stricker and Zigmond 1986). These ideas are captured by statements in the literature such as “Dopamine mediates the ‘working to obtain’ (i.e., tendency to work for motivational stimulus and overcome response constraints, activation for engaging in vigorous  instrumental actions).” (Salamone and Correa 2002, p. 17) or “this dopamine response could assist in preparing the animal to deal with the unexpected by promoting the switching of attentional and behavioral resources” (Redgrave et al. 1999, p. 151) and “functions of the central DA systems could be explained in terms of an ‘energetic’ construct (i.e., one that accounts for the vigor and frequency of behavioral output) of activation.” (Robbins and Everitt 2006, this issue). Those sensorimotor hypotheses have much to recommend them and are supported by substantial evidence. Neuroscientists agree that dopamine systems play roles in movement activation and control and attention and arousal (Albin et al. 1995; Dauer and Przedborski 2003; Redgrave et al. 1999; Salamone and Correa 2002; Salamone et al. 2005). As an example from the 2005 Gordon debate, Salamone and colleagues have convincingly shown that low-dose neuroleptics shift choices away from effortful toward easy tasks, even at the cost of a preferred reward. However, activation-sensorimotor hypotheses are very general in scope, which makes it difficult for them to explain specific aspects of reward. They do not attempt to give clear and specific explanations of why rewards are hedonically pleasant or learned about or sought after. By extension to dopamine’s role in drug addiction and related disorders, they do not attempt to explain why addicts become compulsively motivated to take drugs again. To explain reward-specific aspects of dopamine activation and of addictive drugs, we need hypotheses of dopamine function that address more reward-specific processes themselves. In short, activation, effort or sensorimotor function does not explain why dopamine effects are rewarding, predictive or motivating—even though general activation function may be valid and important. For the rest of this paper, therefore, I will accept that dopamine does have general sensorimotor-activation functions, and will not challenge those hypotheses. But the discussion must move beyond them for the purpose of understanding dopamine’s more specific contributions to reward. We must turn to specific reward hypotheses of what dopamine does.

Analysis of hedonia hypothesis

The hedonia hypothesis suggests that dopamine in nucleus accumbens essentially is a ‘pleasure neurotransmitter’. It was developed chiefly by Roy Wise and his colleagues in the 1970s and 1980s and became a very influential view. As Wise originally put it: “the dopamine junctions represent a synaptic way station...where sensory inputs are translated into the hedonic messages we experience as pleasure, euphoria or ‘yumminess’” (Wise 1980, p. 94). Continuing echoes of the hedonia hypothesis might perhaps still be heard in more recent neuroscience statements such as “Clearly, the mesocorticolimbic dopamine system is critical for psychostimulant activation and psychomotor stimulant reinforcement and plays a role in the reinforcing action of other drugs” (Koob and Le Moal 2006, p. 89) or “The ability of drugs of abuse to increase dopamine in nucleus accumbens underlies their reinforcing effects.” ............................

Well, you get the drift. For the real article, sadly without Angelina, go here.

Searching for Red Stockings: The Myth of Information Overload

As the internet advocate Clay Shirky noted, everybody who talks about information overload starts with the graph with the telltale ascending line and the litany of the troubles it entails. As the line informs us, information is increasing exponentially, and we can barely deal with it intellectually and emotionally, or more and more often, we can’t. And the solution?  It is here that the rallying cries diverge. 

Scary Graph  (from Basex.com)

On one side there is Shirky, who assigns the problem to filter failure, and why not? It’s a reasonable thing after all to suppose that if we had better ways to sort out information, we could cull the bad from the good, and be able to significantly reduce the information we have to cope with daily.  Search, social media, and e-commerce firms of course concur, and are rapidly improving their search algorithms (using of course information about you that you voluntarily or involuntarily port over to them) so  you can find what you need the first time.

On the other hand is the internet critic Nicolas Carr, who attributes information overload to filter success. In Carr’s opinion our filters are working all too well, and the problem is that they are getting better and better. 

Thus,

“….The real source of information overload…. is the stuff we like, the stuff we want. And as filters get better, that's exactly the stuff we get more of.

It's a mistake, in short, to assume that as filters improve they have the effect of reducing the information we have to look at. As today's filters improve, they expand the information we feel compelled to take notice of. Yes, they winnow out the uninteresting stuff (imperfectly), but they deliver a vastly greater supply of interesting stuff. And precisely because the information is of interest to us, we feel pressure to attend to it. As a result, our sense of overload increases.”

Implicit in both arguments is this premise:

The information we want is the same as the information we need.

This is an argument for the curing salve of better filters (to fine tune what we want, since our wants are finite) or a call for mass despair (because our wants are infinite, and thus overwhelm us when they are invariably served by the web). This premise derives from an assumption that in our hubris we are wont to make: that humans are rational agents who know what they want and why.

But what if this was not true?  What if we are at root irrational creatures who delude ourselves into thinking that we know what we want and why we want it? What if the information we want is more often than not different from the information we need? If this is true, then to paraphrase Shakespeare, our fate is not in the stars (or rather the cloud), but in ourselves, because if the information that we want is often not the same as the information we need, then we need to be aware of how to distinguish our wants from our needs and how and when  to constrain the former.  In other words, for information overload, the key is to understand how our basic motivations work.

The question that Shirky and Carr beg is thus elemental: Why is information of interest to us, because it is important, or because of something else?  To answer this question, let us illustrate how a basic search was performed over the last few generations by going to our metaphorical sock drawer in search of red stockings.

It’s 1912, and you as t-shirt manufacturer want to begin a production run of commemorative t-shirts of the Boston Red Stockings triumph in the World Series. As soon as the game is over you receive an immediate telegraph of their victory, and it’s off to the races to start production.

It’s 1932, and you as a t- shirt manufacturer want to get started with your commemorative t-shirt run, and so you listen to the game on the radio, and upon its completion, get to work.

It’s 2012, and you as a t-shirt manufacturer want get to cracking on your production run celebrating the Boston Red Sox victory, and you follow the sox from college draft to preseason to all of their games through the World Series, and monitor all the social and news media who have something to say about it.

In all three time frames, the decision point happens in a second at a predetermined moment, namely when (hopefully) the sox win. The narrative of how that final fact (a sox victory in the final game) got there is irrelevant. No matter what era, the decision point is concise, precise, and momentary, and gets to you on time regardless of the media you use and irrespective of its background story.  There is no need to follow the narrative that describes the changing facts that get us to that point, as the point of the last man flying out in the last inning is all we need.

The difference between the three eras is that in the first era we could not follow the narrative that follows the sox on their way to the pennant, but in the latter era we could, thanks to the rapidly declining transaction cost for information that allows us to perceive the changing flow or narrative of information. But following the latter comes at a cost.  By following the progress of the sox we become diverted from other things of value, and suffer regret.  If these diversions are small scale and populate our working day, they become distractions and cause us to lose focus and attention. Finally, as we continually choose between distraction and staying on course, we become tense and nervous.

The metaphor of ‘information overload’ would seem to apply here, as every frame of every moment of the continuous narrative leading to the Red Sox pennant can and is considered by the sox fan. However, like a strip of static frames in a motion picture that give rise to a sense of movement or motion, the story is interesting because of the novel ways the narrative changes, and it is the changes that compel. Thus, although the ending is necessary for us to go about our business, the story that leads to it is compelling not because of what it is, but how it is continually transformed.

We can expand our simple Red Sox narrative to the narratives embedded in all the things we do that are being progressively revealed by the web. We need to know facts, but what obsesses us is the narrative or story that brings us to those facts.  The internet produces not just more information, but more changing patterns of information. We see not a picture, but a movie, not a note, but a score, not a phrase but a speech. Moreover, we conflate the importance of the narrative with the significance of its conclusion, or what we want with what we need. This is a dangerous delusion, for the stuff we want depends upon the narrative or facts in motion, but the stuff we need depends upon the facts sitting still.

We can get the facts we daily need in a half hour, but continually accessing the web to see a moving stock market, an evolving middle east crisis, or what Uncle Charlie is up to are never ending stories that excite us, engage us, but ultimately bring us down. A narrative is of course still important if our behavior necessarily changes in tandem. In this case the narrative is ‘feedback’. Thus, a quarterback’s performance is determined by feedback during the moment to moment course of the game. However, for the stadium audience, this feedback is entertainment, and for those who attend to the ever expanding narrative on the game itself, an unnecessary and harmful obsession.

The Myth of Information Overload

As a metaphor, information overload attributes the psychological effects of the internet to what information is rather than how it is arranged, and ultimately the metaphor of information overload in inadequate because we are not overloaded with information but with ever evolving novel patterns of information or narratives. Because humans are above all novelty-seeking creatures, novelty is enhanced not in the facts but in the stories they tell. Our interest  lies not only in the rational but in the abstract properties of information. So it is not information that overloads, but elemental aspects of information, namely novelty that when produced in infinite abundance by the web leads us to endless distraction, stress, and regret.  Because the explanation for how the web influences us psychologically is based on core assumptions on human motivation that are faulty, namely that we are rational actors, we proceed with our daily lives under a dangerous illusion abetted unfortunately by the perverse incentives of our media providers to keep us hanging onto the story when the conclusion is all we need. Whether or not we can escape this illusion and its dire consequences depends ultimately on not just a better story, but also a better explanation as to how our minds actually work.

Sources:

Finding a better story to describe the emerging fact that wanting and liking aren’t the same thing takes you to the seminal work performed by the neuropsychologist Kent Berridge on the topic, or my own narrative on Berridge’s work and what it means.  Hopefully both make for some interesting explanations.

Feedback Overload

Since the invention of writing, the written word has literally piled up. Indeed, from very early on, mankind has been overloaded with information. However, the problem posed by of information overload is not in a metaphorical stack of stuff, but in our relative inability of finding the needle of information we need in the haystack of information we don’t. Things like the Dewey decimal system, book indexes, and a helpful librarian barely addressed the problem until the invention of the internet search engine allows us to find our need, or in this case, needles. As the pundit Nicolas Carr opined[i], the problem we confront today is not finding a needle in an infinite informational haystack, but finding an infinite stack of needles that all merit consideration. Nowadays, when we electronically search for any topic, we are provided with many similar bits of information that allow us to more precisely fine tune or correct the deficiencies of knowledge. This error correction or feedback function represents a progressive resolution of the discrepancies between what we do and don’t know. Feedback may represent unexpected changes in our progress to a goal and/or unexpected changes in our knowledge of the nature of a goal. Feedback of course is essential to learning, but consequential to that learning is the increased activity of midbrain dopamine neurons, and it is the neuro-modulator dopamine that enables the consolidation of memory as well as heightened alertness and attention on the task at hand. But dopamine also increases positive affect that adds momentary value or ‘incentive salience’ to behavior, but does not intrinsically predict the overall or long term goodness or utility of behavior. Put a bit differently by the neuro-psychologist Kent Berridge, “The brain results suggest that pure decision utility—and not predicted utility—is raised by activating mesolimbic dopamine systems[ii].”  What this means is that the importance of the decision in the moment, or its ‘decision utility’ does not influence its long term or ‘predicted utility’. The implications of this are profound, for as the marginal utility of examining each informative ‘needle’ declines, the successive needles of information remain novel, and we continue to dwell on nearly redundant links of information not because they are useful but because they are new. In other words, whereas in the past impoverished feedback environments caused us to waste much time looking for information, the rich feedback environments heralded by improvements in web search lead us to waste much time looking at information! This means that we will be affectively and not rationally inclined to overstay our welcome on sites that not only provide us what we want and need, but infinite variations of the same information that we ‘want’ but don’t need. The problem thus is not information overload, but ‘feedback overload’, as the ever increasing amount and granularity of information feedback provides greater and greater detail that can increase the short term or moment to moment value of behavior to the detriment of our long term interests.

Information Search, CA 1960

Information Search, CA 2011

This increase in the momentary incentive salience of behavior can be used to conform with (if not predict) practical ends, but its ultimate value depends upon whose practical ends.  For example, the ‘Khan Academy’ (khanacademy.org) is an online math tutorial that uses rich feedback embodied in badges, scores, hints, etc. to increase the decision utility of performing math exercises in service of the predicted utility of long term mastery of say, the mathematical calculus. On the other hand, a Google search also provides rich feedback including social network feeds, instant messaging, videos, helpful links, and now badges in the service of the predicted utility of Google, namely advertising. 

Ultimately, the problem is not that we are lost in a haystack, but that we are proverbially resting on a bed of pins and needles with each pin needlessly diverting our attention. The notion of ‘feedback overload’ means we are neurologically inclined to overvalue the short term importance or salience of new information, and when new information scales in amount and availability, we begin to live for a moment that may not conform to our ultimate good. For the rich feedback mechanisms provided by the internet, whether it is social media of just plain search, the solution to this problem is not better filtering of information or better feedback (as this merely acerbates the problem), but less, and can only be accomplished by constraining what information you can see, or when you can see it. The simple solution is keeping your personal library and newspaper, and severely restricting your time with search tools (the internet) that work too well. As internet feedback trends to infinity in ever morphing detail and availability, this will be our only option to spare us a new dark age caused by being blinded by the light.


For much more on feedback overload, take a look at my new e-book on the psychology of the internet:

http://www.scribd.com/doc/69880622/One-Track-Minds-The-Surprising-Psychology-of-the-Internet

References:

Berridge, K. and Aldridge, J. W. Decision utility, the brain, and the pursuit of hedonic Goals, Social Cognition, Vol. 26, No. 5, 2008, pp. 621–646

http://www-personal.umich.edu/~berridge/

Nicolas Carr, Roughtype (blog)

http://www.roughtype.com/archives/2011/03/situational_ove.php