Search This Blog

Sunday, July 04, 2004

Flow, Intrinsic Motivation, and 2nd Generation Cognitive Science (2000)

In psychology, decision making, whether it is initiated or motivated by a conscious or non conscious estimation of the contingencies of reinforcement (as behaviorists would see it), biological sensitivities (as sociobiologists would see it), or mentalistic agencies such as will, need, or desire (as pop or social psychologists would see it) is generally accounted for apart from the patterns of neural activation that are responsible for translating motivating events into behavior. The actual neural and somatic processes that activate behavior are generally not considered as having a significant influence on how behavior is decided. Decision making is thus ‘disembodied’, since a consideration of bodily influences would not increase the predictive power bestowed by the behavioral road map provided by environmental contingencies, biological sensitivities, or just good old common sense.

In contrast to the fixation on of behavioral, social, genetic and otherwise disembodied motivating entities that comprise most of what we call the psychology of motivation, cognitive psychology is primarily concerned with the physiological, biochemical, and morphological content of mental states, and how those states are integrated to produce vision, perception, attention, awareness, etc. In other words, cognitive science emphasizes how mental processes are activated, not why they are activated. This bifurcation of psychological perspective into a separate emphasis on motivation and activation variables has historically separated cognitive science from those disciplines in psychology such as analytic, social and behavioristic psychology that are concerned primarily with how behavior is instigated or motivated.

Within the last twenty years however, a new generation of cognitive scientists has begun to regard this divide as artificial, and has recognized that motivation and activation are tightly bound, and that each informs and explains the other. This position, also called bio-behavioral or second generation cognitive science, has found elegant and forceful advocacy in the works of the psychologists John Donahoe and David Palmer, the philosopher Mark Johnson, the linguist George Lakoff, the neuro-psychologist Antonio Damasio, and the immunologist Gerald Edelman. Their general position is that language, behavior, and value itself it strongly rooted in sensori-motor or bodily processes, and that activation in itself does influence motivation. Thus sensori-motor and neural activation patterns underlie basic processes of reinforcement (Donahoe and Palmer, 1993), create the basis of language through metaphor (Johnson and Lakoff, 1999), continually influence behavior as emotion (Damasio, 1994), and provide the basis of value and consciousness (Edelman, 1992).

However, the complex activation patterns that underlie language, emotion, and value do not in themselves implicate how decision making at its most rudimentary is performed. And here is the critical point. If decision making, if reduced to its molecular correlate of activating attentional processes in itself adds value to behavior, then no behavior can be reliably predicted without a consideration of the activation processes that make behavior possible. Moreover, if this is true, then many of the long held concepts in social psychology, behaviorism, and sociobiology must be reconsidered, reinterpreted, or abandoned.

On a molecular (neural) level, the shifting of attention that occurs as initiated by a sudden change in an environment-behavior (perceptual) relationship is isomorphic with the liberation of the neurotransmitter dopamine in synaptic clefts between coactive pre- and post synaptic neurons (Donahoe and Palmer, 1993). The functional role of dopamine is to stabilize active neural representations in the frontal cortex (Durstewitz et al. 1999), to label stimuli with appetitive value (i.e. it feels good), and to provide advance reward information before behavior occurs (Schultz, 1999). Finally, dopamine is released in response to perceived deviations from learned predictions in future reward (Montague, et al. 1994), or is sensitive to the perceived salience of responses and their consequences (i.e. response contingencies). In other words, dopamine acts to make thinking processes more efficient and is essential to the process of reinforcement.

If these hypotheses are true, then specific response contingencies (particularly variable ratio or ‘gambling’ schedules of reinforcement) may be hypothesized to signal frequent and consistent deviations in reward prediction, and these deviations will correlate with the consistent and elevated release of dopamine and the felt experience of elation or pleasure. This prediction has been recently borne out by a recent experiment (Koepp et al. 1998) that demonstrated that the increased release of dopamine occurs during the performance of a goal directed motor task: a video game. This result is underscored by an extensive record of self reports that relate ecstatic states to environmental contingencies that require a rapid attentional shifting between salient cognitive precepts (Csikszentmihalyi, 1993).

A specific response contingency (gambling or video game playing) demands a rapid shifting of attention, but multiple concurrent contingencies involving similar behavior that lead to different rewards may do the same thing. Consider this mind experiment. Will, a third grader, is given a homework assignment to write a school play for a not too exacting teacher. Most likely, Will will not find the task interesting, and turn in some banal creation calculated to achieve a good grade and no more. But what if Will was aware of a cute classmate who swooned over romantic phrases, a principal who liked slapstick, the call of posterity if the play was entered in a nationwide contest, an audience that liked good plot suspense, not to mention writing a better piece than a rival student in class? For better illustration, let’s flesh out this scenario out with a more cinematic example. Will can become William Shakespeare, the principal the Queen, the cute classmate a beautiful countess, his student rival Christopher Marlowe, and posterity the enrapture of audiences for all time. As the movie ‘Shakespeare in Love’ fancifully notes, Shakespeare’s banal play idea of ‘Romeo and Ethel: The Pirates Daughter’ evolved into the resulting Romeo and Juliet due to the influence of a confluence of events or otherwise ‘extrinsic’ motivators. Each of these motivators would in itself have an unremarkable effect on Will, but representing a calculus of ever changing constraints, would cause him to continually shift his attention to succeed in this psychological juggling act. Stressful, perhaps, but exhilarating, definitely.

If activation patterns (attention) are specific to prediction error, then the continuous prediction errors that sustains attention and the corresponding release of dopamine are inherent in the stimulus context that creates that prediction error, and not in the stimulus itself. Thus extrinsic motivators (money, power, sex, etc.) always include an intrinsically motivating component (pleasurable alertness) that is critically dependent upon how those extrinsic events are arranged to follow behavior. Thus extrinsic and intrinsic motivation are not separate entities, but are individual aspects of the same motivating or learning process. This explains the favor we give to behaviors that have a great deal of unpredictability, such as sports, gambling, and perhaps play writing. The leads to a critical hypothesis: Objects do not reinforce, but rather the perceptual relationships denoted by those objects. These relationships occur virtually, and always represent as-if relationships and their corresponding prediction errors. That is, an object does not have to be possessed, but be potentially possessed.

Reinforcement theory, as defined through modern cognitive science, contrasts with theories informed by behaviorism, sociobiology, economics, and even social psychology that correlate happiness with physical objects. Humans do exist to maximize value, but if value is determined by potential rather than actual events, then value can only be maximized by engineering social environments that provide multiple concurrent rewards for differing aspects of behavior, and inculcating through education our ability to perceive those rewards. In other words, perfectly engineered societies are merely the societies we admire anyway, such as the idealized models represented by ancient Greece, Renaissance Italy, and Shakespearean England. These societies provide the multi-modal reinforcement for any variety of endeavor, from art to music to sports, but as importantly, they provide the means whereby we may easily perceive the potential approval of the an audience, a girlfriend, posterity, or a Queen. And the Shakespeares of the world, as well as its musicians, artists and saints come about because they are well reinforced by what they do, and this because they can empathize about the world.



Ashby, F. Gregory, Isen, Alice M. and U. Turken (1999) A Neuropsychological Theory of Positive Affect and Its Influence on Cognition, Psychological Review, 106(3), 529-550

Csikszentmihalyi, Mihaly (1997) Finding Flow, Basic Books

Damasio, Antonio R. (1994) Descartes Error: Emotion, Reason, and the Human Brain. Avon

Deutsch, David (1997) The Fabric of Reality, Allen Lane

Donahoe, J. W. and D. C. Palmer (1993) Learning and Complex Behavior, Allyn and Bacon

Donahoe, J. W., D. C. Palmer, and Jose E. Burgos (1997) The Unit of Selection: What do reinforcers reinforce?, Journal of the Experimental Analysis of Behavior, 67, 259-273

Edelman, Gerald (1992) Bright Air Brilliant Fire, Basic Books

Durstewitz, Daniel, Marian Kelc, and Onur Gunturkun (1999) A neurocomputational theory of the dopaminergic modulation of working memory functions, Journal of Neuroscience, 19(7): 2807-2822

Horvitz, J. C., Stewart, T., and Jacobs, B. L. (1997) Burst activity of ventral segmental dopamine neurons as elicited by sensory stimuli. Brain Research, 759, 251-258

Kuhn, Thomas S. (1962) The Structure of Scientific Revolutions, University of Chicago

Lakoff, George, and Johnson, Mark (1999) Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought, Basic Books

M. J. Koepp et al. (1998) Evidence for striatal dopamine release during a video game, Nature, 393, 266-268

Montague, P.R., Dayan P. Sejnowski, T.J. (1994) Foraging in an uncertain environment using predictive Hebbian learning. In: J.D. Dowan, G. Tesauro and J. Alspector (eds.) Neural Information Processing Systems, 6, Morgan Kaufmann, San Francisco, pp. 598-605

Schultz, Wofram (1998) Predictive reward signal of dopamine neurons, Journal of Neurophysiology, 80, 1, 1-27

Weiner, Norbert, Cybernetics: Control and Communication in the Animal and the Machine. Cambridge, Mass.: MIT Press, 1961

No comments: