1. Abe H, Lee D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70: 731–741, 2011. [PMC free article] [PubMed] [Google Scholar] 2. Adamantidis AR, Tsai HC, Boutrel B, Zhang F, Stuber GD, Budygin EA, Touriño C, Bonci A, Deisseroth K, de Lecea L. Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior. J Neurosci 31: 10829–10835, 2011. [PMC free article] [PubMed] [Google Scholar] 3. Adrian ED, Zotterman Y. The impulses produced by sensory nerve endings. Part 3. Impulses set up by Touch and Pressure. J Physiol 61: 465–483, 1926. [PMC free article] [PubMed] [Google Scholar] 4. Ainslie GW. Impulse control in pigeons. J Exp Anal Behav 21: 485–489, 1974. [PMC free article] [PubMed] [Google Scholar] 5. Ainslie GW. Specious rewards: a behavioral theory of impulsiveness and impulse control. Psych Bull 82: 463–496, 1975. [PubMed] [Google Scholar] 6. Alexander GE, Crutcher MD. Neural representations of the target (goal) of visually guided arm movements in three motor areas of the monkey. J Neurophysiol 64: 164–178, 1990. [PubMed] [Google Scholar] 7. Alexander GE, DeLong MR, Strick PL. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu Rev Neurosci 9: 357–381, 1986. [PubMed] [Google Scholar] 8. Amador N, Schlag-Rey M, Schlag J. Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84: 2166–2170, 2000. [PubMed] [Google Scholar] 9. Ambroggi F, Ishikawa A, Fields HL, Nicola SM. Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59: 648–661, 2008. [PMC free article] [PubMed] [Google Scholar] 10. Amemori KI, Graybiel AM. Localized microstimulation of primate pregenual cingulate cortex induces negative decision-making. Nat Neurosci 15: 776–785, 2012. [PMC free article] [PubMed] [Google Scholar] 11. Amiez C, Joseph JP, Procyk E. Anterior cingulate error-related activity is modulated by predicted reward. Eur J Neurosci 21: 3447–3452, 2005. [PMC free article] [PubMed] [Google Scholar] 12. Amiez C, Joseph JP, Procyk E. Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex 16: 1040–1055, 2006. [PMC free article] [PubMed] [Google Scholar] 13. Aosaki T, Graybiel AM, Kimura M. Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. Science 265: 412–415, 1994. [PubMed] [Google Scholar] 14. Aosaki T, Tsubokawa H, Ishida A, Watanabe K, Graybiel AM, Kimura M. Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning. J Neurosci 14: 3969–3984, 1994. [PMC free article] [PubMed] [Google Scholar] 15. Apicella P, Deffains M, Ravel S, Legallet E. Tonically active neurons in the striatum differentiate between delivery and omission of expected reward in a probabilistic task context. Eur J Neurosci 30: 515–526, 2009. [PubMed] [Google Scholar] 16. Apicella P, Ljungberg T, Scarnati E, Schultz W. Responses to reward in monkey dorsal and ventral striatum. Exp Brain Res 85: 491–500, 1991. [PubMed] [Google Scholar] 17. Apicella P, Ravel S, Deffains M, Legallet E. The role of striatal tonically active neurons in reward prediction error signaling during instrumental task performance. J Neurosci 31: 1507–1515, 2011. [PMC free article] [PubMed] [Google Scholar] 18. Apicella P, Scarnati E, Ljungberg T, Schultz W. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J Neurophysiol 68: 945–960, 1992. [PubMed] [Google Scholar] 19. Argilli E, Sibley DR, Malenka RC, England PM, Bonci A. Mechanism and time course of cocaine-induced long-term potentiation in the ventral tegmental area. J Neurosci 28: 9092–9100, 2008. [PMC free article] [PubMed] [Google Scholar] 20. Arsenault JT, Rima S, Stemmann H, Vanduffel W. Role of the primate ventral tegmental area in reinforcement and motivation. Curr Biol 24: 1347–1353, 2014. [PMC free article] [PubMed] [Google Scholar] 21. Asaad WF, Eskandar EN. Encoding of both positive and negative reward prediction errors by neurons of the primate lateral prefrontal cortex and caudate nucleus. J Neurosci 31: 17772–17787, 2011. [PMC free article] [PubMed] [Google Scholar] 22. Aston-Jones G, Rajkowski J, Kubiak P, Alexinsky T. Locus coeruleus neurons in monkey are selectively activated by attended cues in a vigilance task. J Neurosci 14: 4467–4480, 1994. [PMC free article] [PubMed] [Google Scholar] 23. Azzi JCB, Sirigu A, Duhamel JR. Modulation of value representation by social context in the primate orbitofrontal cortex. Proc Natl Acad Sci USA 109: 2126–2131, 2012. [PMC free article] [PubMed] [Google Scholar] 24. Badman MK, Flier JS. The gut and energy balance: visceral allies in the obesity wars. Science 307: 1909–1914, 2005. [PubMed] [Google Scholar] 25. Báez-Mendoza R, Harris C, Schultz W. Activity of striatal neurons reflects social action and own reward. Proc Natl Acad Sci USA 110: 16634–16639, 2013. [PMC free article] [PubMed] [Google Scholar] 26. Balleine B, Dickinsion A. Goal-directed instrumental action: contingency, and incentive learning and their cortical substrates. Neuropharmacology 37: 407–419, 1998. [PubMed] [Google Scholar] 27. Bangasser DA, Waxler DE, Santollo J, Shors TJ. Trace conditioning and the hippocampus: the importance of contiguity. J Neurosci 26: 8702–8706, 2006. [PMC free article] [PubMed] [Google Scholar] 28. Bao S, Chan VT, Merzenich MM. Cortical remodelling induced by activity of ventral tegmental dopamine neurons. Nature 412: 79–83, 2001. [PubMed] [Google Scholar] 29. Barnes TD, Kubota Y, Hu D, Jin DZ, Graybiel AM. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437: 1158–1161, 2005. [PubMed] [Google Scholar] 30. Barraclough D, Conroy ML, Lee DJ. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7: 405–410, 2004. [PubMed] [Google Scholar] 31. Barto AG, Singh S, Chentanez N. Intrinsically motivated learning of hierarchical collections of skills. In: Advances in Neural Information Processing Systems 17: Proceedings of the 2004 Conference. Cambridge, MA: MIT Press, 2005. [Google Scholar] 32. Barto AG, Sutton RS, anderson CW. Neuronlike adaptive elements that can solve difficult learning problems. IEEE Trans Syst Man Cybernet 13: 834–846, 1983. [Google Scholar] 33. Bayer HM, Glimcher PW. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47: 129–141, 2005. [PMC free article] [PubMed] [Google Scholar] 34. Bayer HM, Lau B, Glimcher PW. Statistics of dopamine neuron spike trains in the awake primate. J Neurophysiol 98: 1428–1439, 2007. [PubMed] [Google Scholar] 35. Bechara A, Damasio AR, Damasio H, anderson SW. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50: 7–15, 1994. [PubMed] [Google Scholar] 36. Belova MA, Paton JJ, Morrison SE, Salzman CD. Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55, 970–984, 2007. [PMC free article] [PubMed] [Google Scholar] 37. Belova MA, Paton JJ, Salzman CD. Moment-to-moment tracking of state value in the amygdala. J Neurosci 28: 10023–10030, 2008. [PMC free article] [PubMed] [Google Scholar] 38. Bennur S, Gold JI. Distinct representations of a perceptual decision and the associated oculomotor plan in the monkey lateral intraparietal area. J Neurosci 31: 913–921, 2011. [PMC free article] [PubMed] [Google Scholar] 39. Bentham J. An Introduction to the Principle of Morals and Legislations, 1789. Reprinted Oxford, UK: Blackwell, 1948. [Google Scholar] 40. Bermudez MA, Göbel C, Schultz W. Sensitivity to temporal reward structure in amygdala neurons. Curr Biol 22: 1839–1844, 2012. [PMC free article] [PubMed] [Google Scholar] 41. Bermudez MA, Schultz W. Responses of amygdala neurons to positive reward predicting stimuli depend on background reward (contingency) rather than stimulus-reward pairing (contiguity). J Neurophysiol 103: 1158–1170, 2010. [PMC free article] [PubMed] [Google Scholar] 42. Bermudez MA, Schultz W. Reward magnitude coding in primate amygdala neurons. J Neurophysiol 104: 3424–2432, 2010. [PMC free article] [PubMed] [Google Scholar] 43. Bernoulli D. Specimen theoriae novae de mensura sortis. Comentarii Academiae Scientiarum Imperialis Petropolitanae (Papers Imp Acad Sci St Petersburg) 5: 175–192, 1738 (translated as Exposition of a new theory on the measurement of risk. Econometrica 22: 23–36, 1954). [Google Scholar] 44. Berridge KC. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology 191: 391–431, 2007. [PubMed] [Google Scholar] 45. Berridge KC, Robinson TE. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Rev 28: 309–369, 1998. [PubMed] [Google Scholar] 46. Berthoud HR, Morrison C. The brain, appetite, obesity. Annu Rev Psychol 59: 55–92, 2008. [PubMed] [Google Scholar] 47. Beylin AV, Gandhi CC, Wood GE, Talk AC, Matzel LD, Shors TJ. The role of the hippocampus in trace conditioning: temporal discontinuity or task difficulty? Neurobiol Learn Mem 76: 447–461, 2001. [PubMed] [Google Scholar] 48. Bi GQ, Poo MM. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 18: 10464–10472, 1998. [PMC free article] [PubMed] [Google Scholar] 49. Bichot NP, Schall JD. Effects of similarity and history on neural mechanisms of visual selection. Nat Neurosci 2: 549–554, 1999. [PubMed] [Google Scholar] 50. Binmore K, Shaked A. Experimental economics: where next? J Econ Behav Organiz 73: 87–100, 2010. [Google Scholar] 51. Black RW. Shifts in magnitude of reward and contrast effects in instrumental and selective learning: a reinterpretation. Psychol Rev 75: 114–126, 1968. [PubMed] [Google Scholar] 52. Blanchard TC, Wolfe LS, Vlaev I, Winston JS, Hayden BY. Biases in preferences for sequences of outcomes in monkeys. Cognition 130: 289–299, 2014. [PMC free article] [PubMed] [Google Scholar] 53. Blatter K, Schultz W. Rewarding properties of visual stimuli. Exp Brain Res 168: 541–546, 2006. [PubMed] [Google Scholar] 54. Blythe SN, Wokosin D, Atherton JF, Bevan MD. Cellular mechanisms underlying burst firing in substantia nigra dopamine neurons. J Neurosci 29: 15531–15541, 2009. [PMC free article] [PubMed] [Google Scholar] 55. Bogacz R. Optimal decision-making theories: linking neurobiology with behaviour. Trends Cog Sci 11: 118–125, 2007. [PubMed] [Google Scholar] 56. Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol Rev 113: 700–765, 2006. [PubMed] [Google Scholar] 57. Bogacz R, Gurney K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neur Comput 19: 442–477, 2007. [PubMed] [Google Scholar] 58. Bouret S, Richmond BJ. Ventromedial and orbital prefrontal neurons differentially encode internally and externally driven motivational values in monkeys. J Neurosci 30: 8591–8601, 2010. [PMC free article] [PubMed] [Google Scholar] 59. Bouton ME. Context and behavioural processes in extinction. Learning Memory 11: 485–494, 2004. [PubMed] [Google Scholar] 60. Bowman EM, Aigner TG, Richmond BJ. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J Neurophysiol 75: 1061–1073, 1996. [PubMed] [Google Scholar] 61. Bredfeldt CE, Ringach DL. Dynamics of spatial frequency tuning in macaque V1. J Neurosci 22: 1976–1984, 2002. [PMC free article] [PubMed] [Google Scholar] 62. Breton YA, James C, Marcus JC, Shizgal P. Rattus psychologicus: construction of preferences by self-stimulating rats. Behav Brain Res 202: 77–91, 2009. [PubMed] [Google Scholar] 63. Brzezniak Z, Capinski M, Flandoli F. Pathwise global attractors for stationary random dynamical systems. Probab Theory Relat Fields 95: 87–102, 1993. [Google Scholar] 64. Brischoux F, Chakraborty S, Brierley DI, Ungless MA. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci USA 106: 4894–4899, 2009. [PMC free article] [PubMed] [Google Scholar] 65. Brody CD, Hernandez A, Zainos A, Romo R. Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. Cereb Cortex 13: 1196–1207, 2003. [PubMed] [Google Scholar] 66. Bromberg-Martin ES, Hikosaka O. Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron 63: 119–126, 2009. [PMC free article] [PubMed] [Google Scholar] 67. Bromberg-Martin ES, Hikosaka O. Lateral habenula neurons signal errors in the prediction of reward information. Nat Neurosci 14: 1209–1216, 2011. [PMC free article] [PubMed] [Google Scholar] 68. Bromberg-Martin ES, Matsumoto M, Hon S, Hikosaka O. A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104: 1068–1076, 2010. [PMC free article] [PubMed] [Google Scholar] 69. Brosnan SF, de Waal FBM. Monkeys reject unequal pay. Nature 425: 297–299, 2003. [PubMed] [Google Scholar] 70. Brown JR, Arbuthnott GW. The electrophysiology of dopamine (D2) receptors: a study of the actions of dopamine on corticostriatal transmission. Neuroscience 10: 349–355, 1983. [PubMed] [Google Scholar] 71. Brown JW, Bullock D, Grossberg S. How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J Neurosci 19: 10502–10511, 1999. [PMC free article] [PubMed] [Google Scholar] 72. Brown JW, Bullock D, Grossberg S. How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades. Neural Networks 17: 471–510, 2004. [PubMed] [Google Scholar] 73. Brown MTC, Henny P, Bolam JP, Magill PJ. Activity of neurochemically heterogeneous dopaminergic neurons in the substantia nigra during spontaneous and driven changes in brain state. J Neurosci 29: 2915–2925, 2009. [PMC free article] [PubMed] [Google Scholar] 74. Bruce CJ, Goldberg ME. Primate frontal eye fields. I. Single neurons discharging before saccades. J Neurophysiol 53: 603–635, 1985. [PubMed] [Google Scholar] 75. Budygin EA, Park J, Bass CE, Grinevich VP, Bonin KD, Wightman RM. Aversive stimulus differentially triggers subsecond dopamine release in reward regions. Neuroscience 201: 331–337, 2012. [PMC free article] [PubMed] [Google Scholar] 76. Bunney BS, Grace AA. Acute and chonic haloperidol treatment: comparison of effects on nigral dopaminergic cell activity. Life Sci 23: 1715–1728, 1978. [PubMed] [Google Scholar] 77. Burkart JM, Fehr E, Efferson C, van Schaik CP. Other-regarding preferences in a non-human primate: common marmosets provision food altruistically. Proc Natl Acad Sci USA 104: 19762–19766, 2007. [PMC free article] [PubMed] [Google Scholar] 78. Caggiano V, Fogassi L, Rizzolatti G, Casile A, Giese MA, Thier P. Mirror neurons encode the subjective value of an observed action. Proc Natl Acad Sci USA 109: 11848–11853, 2012. [PMC free article] [PubMed] [Google Scholar] 79. Cai X, Kim S, Lee D. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69: 170–182, 2011. [PMC free article] [PubMed] [Google Scholar] 80. Cai X, Padoa-Schioppa C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J Neurosci 32: 3791–3808, 2012. [PMC free article] [PubMed] [Google Scholar] 81. Cai X, Padoa-Schioppa C. Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation. Neuron 81: 1140–1151, 2014. [PMC free article] [PubMed] [Google Scholar] 82. Calabresi P, Gubellini P, Centonze D, Picconi B, Bernardi G, Chergui K, Svenningsson P, Fienberg AA, Greengard P. Dopamine and cAMP-regulated phosphoprotein 32 kDa controls both striatal long-term depression and long-term potentiation, opposing forms of synaptic plasticity. J Neurosci 20: 8443–8451, 2000. [PMC free article] [PubMed] [Google Scholar] 83. Camerer CF. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton, NJ: Princeton Univ. Press, 2003. [Google Scholar] 84. Camille N, Tsuchida A, Fellows LK. Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J Neurosci 31: 15048–15052, 2001. [PMC free article] [PubMed] [Google Scholar] 85. Caplin A, Dean M, Martin D. Search and satisficing. Am Econ Rev 101: 2899–2922, 2011. [Google Scholar] 86. Caraco T, Blankenhorn WU, Gregory GM, Newman JA, Recer GM, Zwicker SM. Risk-sensitivity: ambient temperature affects foraging choice. Anim Behav 39: 338–345, 1990. [Google Scholar] 87. Caraco T, Martindale S, Whitham TS. An empirical demonstration of risk-sensitive foraging preferences. Anim Behav 28: 820–830, 1980. [Google Scholar] 88. Cardinal RN, Howes NJ. Effects of lesions of the nucleus accumbens core on choice between small certain and large uncertain rewards in rats. BMC Neurosci 6: 37, 2005. [PMC free article] [PubMed] [Google Scholar] 89. Chang SWC, Gariépy JF, Platt ML. Neuronal reference frames for social decisions in primate frontal cortex. Nat Neurosci 16: 243–250, 2013. [PMC free article] [PubMed] [Google Scholar] 90. Chang SWC, Winecoff AA, Platt ML. Vicarious reinforcement in rhesus macaques (Macaca mulatta). Front Neurosci 5: 27, 2011. [PMC free article] [PubMed] [Google Scholar] 91. Chen MK, Lakshminarayanan V, Santos LR. How basic are behavioral biases? Evidence from capuchin trading behavior. J Pol Econ 114: 517–537, 2006. [Google Scholar] 92. Chesselet MF. Presynaptic regulation of neurotransmitter release in the brain: facts and hypothesis. Neuroscience 12: 347–375, 1984. [PubMed] [Google Scholar] 93. Chevalier G, Vacher S, Deniau JM, Desban M. Disinhibition as a basic process in the expression of striatal functions. I. The striato-nigral influence on tecto-spinal/tecto-diencephalic neurons. Brain Res 334: 215–226, 1985. [PubMed] [Google Scholar] 94. Chow CC, White JA. Spontaneous action potentials due to channel fluctuations. Biophys J 71: 3013–3023, 2000. [PMC free article] [PubMed] [Google Scholar] 95. Christoph GR, Leonzio RJ, Wilcox KS. Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat. J Neurosci 6: 613–619, 1986. [PMC free article] [PubMed] [Google Scholar] 96. Churchland AK, Kiani R, Chaudhuri R, Wang XJ, Pouget A, Shadlen MN. Variance as a signature of neural computations during decision making. Neuron 69: 818–831, 2011. [PMC free article] [PubMed] [Google Scholar] 97. Churchland AK, Kiani R, Shadlen MN. Decision-making with multiple alternatives. Nat Neurosci 11: 693–702, 2008. [PMC free article] [PubMed] [Google Scholar] 98. Cisek P, Kalaska JF. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45: 801–814, 2005. [PubMed] [Google Scholar] 99. Cisek P, Kalaska JF. Neural mechanisms for interacting with a world full of action choices. Annu Rev Neurosci 33: 269–298, 2010. [PubMed] [Google Scholar] 100. Clarke HF, Dalley JW, Crofts HS, Robbins TW, Roberts AC. Cognitive inflexibility after prefrontal serotonin depletion. Science 304: 878–880, 2004. [PubMed] [Google Scholar] 101. Coe B, Tomihara K, Matsuzawa M, Hikosaka O. Visual and anticipatory bias in three cortical eye fields of the monkey during an adaptive decision-making task. J Neurosci 22: 5081–5090, 2002. [PMC free article] [PubMed] [Google Scholar] 102. Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482: 85–88, 2012. [PMC free article] [PubMed] [Google Scholar] 103. Corbett D, Wise RA. Intracranial self-stimulation in relation to the ascending dopaminergic systems of the midbrain: a moveable microelectrode study. Brain Res 185: 1–15, 1980. [PubMed] [Google Scholar] 104. Crespi LP. Quantitative variation in incentive and performance in the white rat. Am J Psychol 40: 467–517, 1942. [Google Scholar] 105. Critchley HG, Rolls ET. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J Neurophysiol 75: 1673–1686, 1996. [PubMed] [Google Scholar] 106. Cromwell HC, Hassani OK, Schultz W. Relative reward processing in primate striatum. Exp Brain Res 162: 520–525, 2005. [PubMed] [Google Scholar] 107. Cromwell HC, Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89: 2823–2838, 2003. [PubMed] [Google Scholar] 108. Croxson PL, Walton ME, O'Reilly JX, Behrens TEJ, Rushworth MFS. Effort-based cost-benefit valuation and the human brain. J Neurosci 29: 4531–4541, 2009. [PMC free article] [PubMed] [Google Scholar] 109. Cui H, andersen RA. Posterior parietal cortex encodes autonomously selected motor plans. Neuron 56: 552–559, 2007. [PMC free article] [PubMed] [Google Scholar] 110. d'Acremont M, Fornari E, Bossaerts P. Activity in inferior parietal and medial prefrontal cortex signals the accumulation of evidence in a probability learning task. PLoS Comput Biol 9: e1002895, 2013. [PMC free article] [PubMed] [Google Scholar] 111. d'Ardenne K, McClure SM, Nystrom LE, Cohen JD. BOLD Responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319: 1264–1267, 2008. [PubMed] [Google Scholar] 112. Darwin C. On the Origin of Species by Natural Selection, or the Preservation of Favoured Races in the Struggle for Life. London: John Murray, 1859. [PMC free article] [PubMed] [Google Scholar] 113. Daw ND, Courville AC, Touretsky DS. Representation and timing in theories of the dopamine system. Neur Comput 18: 1637–1677, 2006. [PubMed] [Google Scholar] 114. Dawkins R. The Selfish Gene. London: Oxford Univ. Press, 1976. [Google Scholar] 115. Day JJ, Jones JL, Carelli RM. Nucleus accumbens neurons encode predicted and ongoing reward costs in rats. Eur J Neurosci 33: 308–321, 2011. [PMC free article] [PubMed] [Google Scholar] 116. Day JJ, Jones JL, Wightman RM, Carelli RM. Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biol Psychiat 68: 306–309, 2010. [PMC free article] [PubMed] [Google Scholar] 117. Day JJ, Roitman MF, Wightman RM, Carelli RM. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10: 1020–1028, 2007. [PubMed] [Google Scholar] 118. De Araujo IE, Oliveira-Maia AJ, Sotnikova TD, Gainetdinov RR, Caron MC, Nicolelis MAL, Simon SA. Food reward in the absence of taste receptor signaling. Neuron 57: 930–941, 2008. [PubMed] [Google Scholar] 119. De Araujo IE, Rolls ET, Velazco MI, Margot C, Cayeux I. Cognitive modulation of olfactory processing. Neuron 46: 671–679, 2005. [PubMed] [Google Scholar] 120. De Lafuente O, Romo R. Neuronal correlates of subjective sensory experience. Nat Neurosci 8: 1698–1703, 2005. [PubMed] [Google Scholar] 121. De Lafuente O, Romo R. Neural correlate of subjective sensory experience gradually builds up across cortical areas. Proc Natl Acad Sci USA 103: 14266–14271, 2006. [PMC free article] [PubMed] [Google Scholar] 122. De Lafuente O, Romo R. Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions. Proc Natl Acad Sci USA 49: 19767–19771, 2011. [PMC free article] [PubMed] [Google Scholar] 123. De Waal FB, Davis JM. Capuchin cognitive ecology: cooperation based on projected returns. Neuropsychologia 41: 221–228, 2003. [PubMed] [Google Scholar] 124. Deaner RO, Khera AV, Platt ML. Monkeys pay per view: adaptive valuation of social images by rhesus monkeys. Curr Biol 15: 543–548, 2005. [PubMed] [Google Scholar] 125. Deco G, Rolls ET, Romo R. Stochastic dynamics as a principle of brain function. Prog Neurobiol 88: 1–16, 2009. [PubMed] [Google Scholar] 126. Delamater AR. Outcome selective effects of intertrial reinforcement in a Pavlovian appetitive conditioning paradigm with rats. Anim Learn Behav 23: 31–39, 1995. [Google Scholar] 127. Deniau JM, Chevalier G. Disinhibition as a basic process in the expression of striatal function. II. The striato-nigral influence on thalamocortical cells of the ventromedial thalamic nucleus. Brain Res 334: 227–233, 1985. [PubMed] [Google Scholar] 128. Dennett DC. Elbow Room. The Varieties of Free Will Worth Wanting. Boston: MIT Press, 1984. [Google Scholar] 129. Destexhe A, Contreras D. Neuronal computations with stochastic network states. Science 314, 85–88, 2006. [PubMed] [Google Scholar] 130. Di Ciano P, Cardinal RN, Cowell RA, Little SJ, Everitt B. Differential involvement of NMDA, AMPA/kainate, and dopamine receptors in the nucleus accumbens core in the acquisition and performance of Pavlovian approach behavior. J Neurosci 21: 9471–77, 2001. [PMC free article] [PubMed] [Google Scholar] 131. Di Loreto S, Florio T, Scarnati E. Evidence that a non-NMDA receptor is involved in the excitatory pathway from the pedunculopontine region to nigrostriatal dopaminergic neurons. Exp Brain Res 89: 79–86, 1992. [PubMed] [Google Scholar] 132. Dickinson A. Contemporary Animal Learning Theory. Cambridge, UK: Cambridge Univ. Press, 1980, p. 43. [Google Scholar] 133. Dickinson A, Balleine B. Motivational control of goal-directed action. Anim Learn Behav 22: 1–18, 1994. [Google Scholar] 134. Ding L, Gold JI. Caudate encodes multiple computations for perceptual decisions. J Neurosci 30: 15747–15759, 2010. [PMC free article] [PubMed] [Google Scholar] 135. Ding L, Hikosaka O. Comparison of reward modulation in the frontal eye field and caudate of the macaque. J Neurosci 26: 6695–6703, 2006. [PMC free article] [PubMed] [Google Scholar] 136. Dormont JF, Conde H, Farin D. The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. I. Context-dependent and reinforcement-related single unit activity. Exp Brain Res 121: 401–10, 1998. [PubMed] [Google Scholar] 137. Dorris MC, Glimcher PW. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44: 365–378, 2004. [PubMed] [Google Scholar] 138. Doucet G, Descarries L, Garcia S. Quantification of the dopamine innervation in adult rat neostriatum. Neuroscience 19: 427–445, 1986. [PubMed] [Google Scholar] 139. Doya K, Samejima K, Katagiri K, Kawato M. Multiple model-based reinforcement learning. Neural Comput 14: 1347–1369, 2002. [PubMed] [Google Scholar] 140. Drevets WC, Gautier C, Price JC, Kupfer DJ, Kinahan PE, Grace AA, Price JL, Mathis CA. Amphetamine-induced dopamine release in human ventral striatum correlates with euphoria. Biol Psychiatry 49: 81–96, 2001. [PubMed] [Google Scholar] 141. Eblen F, Graybiel AM. Highly restricted origin of prefrontal cortical inputs to striosomes in the macaque monkey. J Neurosci 15: 5999–6013, 1995. [PMC free article] [PubMed] [Google Scholar] 142. Edgeworth F. Mathematical Psychics: An Essay on the Application of Mathematics to the Moral Sciences. New York: Augustus M. Kelly, 1881. [Google Scholar] 143. Ekman P. An argument for basic emotions. Cogn Emot 6: 169–200, 1992. [Google Scholar] 144. Enomoto K, Matsumoto N, Nakai S, Satoh T, Sato TK, Ueda Y, Inokawa H, Haruno M, Kimura M:. Dopamine neurons learn to encode the long-term value of multiple future rewards. Proc Natl Acad Sci USA 108: 15462–15467, 2011. [PMC free article] [PubMed] [Google Scholar] 145. Estes WK. Discriminative conditioning. I: a discriminative property of conditioned anticipation. J Exp Psychol 32: 150–155, 1943. [Google Scholar] 146. Everitt BJ, Stacey P. Studies of instrumental behavior with sexual reinforcement in male rats (Rattus norvegicus). II. Effects of preoptic area lesions, castration, and testosterone. J Com Psychol 101: 407–419, 1987. [PubMed] [Google Scholar] 147. Everling S, Munoz DP. Neuronal correlates for preparatory set associated with pro-saccades and anti-saccades in the primate frontal eye field. J Neurosci 20: 387–400, 2000. [PMC free article] [PubMed] [Google Scholar] 148. Fairhall AL, Lewen GD, Bialek W, de Ruyter van Steveninck RR. Efficiency and ambiguity in an adaptive neural code. Nature 412: 787, 2001. [PubMed] [Google Scholar] 149. Fehr E, Camerer CF. Social neuroeconomics: the neural circuitry of social preferences. Trends Cogn Neurosci 11: 419–427, 2007. [PubMed] [Google Scholar] 150. Fehr E, Schmidt KM. A theory of fairness, competition, and cooperation. Q J Econ 114: 817–688, 1999. [Google Scholar] 151. Fehr E, Schmidt KM. On inequity aversion: a reply to Binmore and Shaked. J Econ Behav Organiz 73: 101–108, 2010. [Google Scholar] 152. Fehr-Duda H, Bruhin A, Epper T, Schubert R. Rationality on the rise: Why relative risk aversion increases with stake size. J Risk Uncertain 40: 147–180, 2010. [Google Scholar] 153. Feierstein CE, Quirk MC, Uchida N, Sosulski DL, Mainen ZF. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51: 495–507, 2006. [PubMed] [Google Scholar] 154. Fiala J, Grossberg S, Bullock D. Metabotropic glutamate receptor activation in cerebellar purkinje cells as substrate for adaptive timing of the classically conditioned eye-blink response. J Neurosci 16: 3760–3774, 1996. [PMC free article] [PubMed] [Google Scholar] 155. Fibiger HC, LePiane FG, Jakubovic A, Phillips AG. The role of dopamine in intracranial self-stimulation of the ventral tegmental area. J Neurosci 7: 3888–3896, 1987. [PMC free article] [PubMed] [Google Scholar] 156. Fiorillo CD. Transient activation of midbrain dopamine neurons by reward risk. Neuroscience 197: 162–171, 2011. [PMC free article] [PubMed] [Google Scholar] 157. Fiorillo CD. Two dimensions of value: dopamine neurons represent reward but not aversiveness. Science 341: 546–549, 2013. [PubMed] [Google Scholar] 158. Fiorillo CD, Newsome WT, Schultz W. The temporal precision of reward prediction in dopamine neurons. Nat Neurosci 11: 966–973, 2008. [PubMed] [Google Scholar] 159. Fiorillo CD, Song MR, Yun SR. Diversity and homogeneity in responses of midbrain dopamine neurons. J Neurosci 33: 4693–4709, 2013. [PMC free article] [PubMed] [Google Scholar] 160. Fiorillo CD, Song MR, Yun SR. Multiphasic temporal dynamics in responses of midbrain dopamine neurons to appetitive and aversive stimuli. J Neurosci 33: 4710–4725, 2013. [PMC free article] [PubMed] [Google Scholar] 161. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299: 1898–1902, 2003. [PubMed] [Google Scholar] 162. Fiorillo CD, Tobler PN, Schultz W. Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than backpropagating TD errors. Behav Brain Funct 1: 7, 2005. [PMC free article] [PubMed] [Google Scholar] 163. Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PE, Akil H. A selective role for dopamine in stimulus-reward learning. Nature 469: 53–57, 2011. [PMC free article] [PubMed] [Google Scholar] 164. Flaherty AW, Graybiel A. Output architecture of the primate putamen. J Neurosci 13: 3222–3237, 1993. [PMC free article] [PubMed] [Google Scholar] 165. Fliessbach K, Weber B, Trautner P, Dohmen T, Sunde U, Elger CE, Falk A. Social comparison affects reward-related brain activity in the human ventral striatum. Science 318: 1305–1308, 2007. [PubMed] [Google Scholar] 166. Floresco SB, West AR, Ash B, Moore H, Grace AA. Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat Neurosci 6: 968–973, 2003. [PubMed] [Google Scholar] 167. Fourcaud-Trocme N, Brunel N. Dynamics of the firing probability of noisy integrate-and-fire neurons. Neur Comp 14: 2057–2110, 2002. [PubMed] [Google Scholar] 168. Fox RF, Lu Y. Emergent collective behavior in large numbers of globally coupled independently stochastic ion channels. Phys Rev E 49: 3421–3425, 1994. [PubMed] [Google Scholar] 169. Frank MJ, Seeberger LC, O'Reilly RC. By carrot or by stick: Cognitive reinforcement learning in parkinsonism. Science 306: 1940–1943, 2004. [PubMed] [Google Scholar] 170. Frémaux N, Sprekeler H, Gerstner W. Functional requirements for reward-modulated spike-timing-dependent plasticity. J Neurosci 30: 13326–13337, 2010. [PMC free article] [PubMed] [Google Scholar] 171. Freund TF, Powell JF, Smith AD. Tyrosine hydroxylase-immunoreactive boutons in synaptic contact with identified striatonigral neurons, with particular reference to dendritic spines. Neuroscience 13: 1189–1215, 1984. [PubMed] [Google Scholar] 172. Fried I, Mukamel R, Kreiman G. Internally generated preactivation of single neurons in human medial frontal cortex predicts volition. Neuron 69: 548–562, 2011. [PMC free article] [PubMed] [Google Scholar] 173. Friedman M, Savage LJ. The utility analysis of choices involving risk. J Polit Econ 56: 279–304, 1948. [Google Scholar] 174. Gaffan D, Murray EA, Fabre-Thorpe M. Interaction of the amygdala with the frontal lobe in reward memory. Eur J Neurosci 5: 968–975, 1993. [PubMed] [Google Scholar] 175. Gallistel CR, Gibbon J. Time, rate and conditioning. Psych Rev 107: 289–344, 2000. [PubMed] [Google Scholar] 176. Gao WY, Lee TH, King GR, Ellinwood EH. Alterations in baseline activity and quinpirole sensitivity in putative dopamine neurons in the substantia nigra and ventral tegmental area after withdrawal from cocaine pretreatment. Neuropsychopharmacology 18: 222–232, 1998. [PubMed] [Google Scholar] 177. Gauthier J, Parent M, Lévesque M, Parent A. The axonal arborization of single nigrostriatal neurons in rats. Brain Res 834: 228–232, 1999. [PubMed] [Google Scholar] 178. Gdowski MJ, Miller LE, Parrish T, Nenonene EK, Houk JC. Context dependency in the globus pallidus internal segment during targeted arm movements. J Neurophysiol 85: 998–1004, 2001. [PubMed] [Google Scholar] 179. Genovesio A, Tsujimoto S, Navarra G, Falcone R, Wise SP. Autonomous encoding of irrelevant goals and outcomes by prefrontal cortex neurons. J Neurosci 34: 1970–1978, 2014. [PMC free article] [PubMed] [Google Scholar] 180. Gerstein G, Mandelbrot B. Random walk models for the spike activity of a single neuron. Biophys J 4: 41–68, 1964. [PMC free article] [PubMed] [Google Scholar] 181. Gietzen DW, Hao S, Anthony TG. Mechanisms of food intake repression in indispensable amino acid deficiency. Annu Rev Nutr 27: 63–78, 2007. [PubMed] [Google Scholar] 182. Gilbert PFC, Thach WT. Purkinje cell activity during motor learning. Brain Res 128: 309–328, 1977. [PubMed] [Google Scholar] 183. Glimcher PW. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc Natl Acad Sci USA 108: 15647–15654, 2011. [PMC free article] [PubMed] [Google Scholar] 184. Gnadt JW, andersen RA. Memory related motor planning activity in posterior parietal cortex of macaque. Exp Brain Res 70: 216–220, 1988. [PubMed] [Google Scholar] 185. Gold JI, Shadlen MN. The neural basis of decision making. Annu Rev Neurosci 30: 535–574, 2007. [PubMed] [Google Scholar] 186. Goldman-Rakic PS, Leranth C, Williams MS, Mons N, Geffard M. Dopamine synaptic complex with pyramidal neurons in primate cerebral cortex. Proc Natl Acad Sci USA 86: 9015–9019, 1989. [PMC free article] [PubMed] [Google Scholar] 187. Gonzalez R, Wu G. On the shape of the probability weighting function. Cogn Psychol 38: 129–166, 1999. [PubMed] [Google Scholar] 188. Grabenhorst F, Hernadi I, Schultz W. Prediction of economic choice by primate amygdala neurons. Proc Natl Acad Sci USA 109: 18950–18955, 2012. [PMC free article] [PubMed] [Google Scholar] 189. Grabenhorst F, Rolls ET, Bilderbeck A. How cognition modulates affective responses to taste and flavor: top-down influences on the orbitofrontal and pregenual cingulate cortices. Cereb Cortex 18: 1549–1559, 2008. [PubMed] [Google Scholar] 190. Grabenhorst F, Rolls ET, Margot C, da Silva MAAP, Velazco MI. How pleasant and unpleasant stimuli combine in different brain regions: odor mixtures. J Neurosci 27: 13532–13540, 2007. [PMC free article] [PubMed] [Google Scholar] 191. Grether WF. Pseudo-conditioning without paired stimulation encountered in attempted backward conditioning. Comp Psychol 25: 91–96, 1938. [Google Scholar] 192. Groves PM, Garcia-Munoz M, Linder JC, Manley MS, Martone ME, Young SJ. Elements of the intrinsic organization and information processing in the neostriatum. In: Models of Information Processing in the Basal Ganglia, edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press, 1995, p. 51–96. [Google Scholar] 193. Guarraci FA, Kapp BS. An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit. Behav Brain Res 99: 169–179, 1999. [PubMed] [Google Scholar] 194. Gurden H, Takita M, Jay TM. Essential role of D1 but not D2 receptors in the NMDA receptor-dependent long-term potentiation at hippocampal-prefrontal cortex synapses in vivo. J Neurosci 106: 1–5, 2000. [PMC free article] [PubMed] [Google Scholar] 195. Gutkin BS, Ermentrout GB. Dynamics of membrane excitability determine interspike interval variability: a link between spike generation mechanisms, and cortical spike train statistics. Neur Comp 10: 1047–1065, 1998. [PubMed] [Google Scholar] 196. Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20: 2369–2382, 2000. [PMC free article] [PubMed] [Google Scholar] 197. Haber SN, Kim KS, Mailly P, Calzavara R. Reward-related cortical inputs define a large striatal region in primates that interface with associative cortical connections, providing a substrate for incentive-based learning. J Neurosci 26: 8368–8376, 2006. [PMC free article] [PubMed] [Google Scholar] 198. Hanes DP, Schall JP. Neural control of voluntary movement initiation. Science 274: 427–430, 1996. [PubMed] [Google Scholar] 199. Hanks T, Kiani R, Shadlen MN. A neural mechanism of speed-accuracy tradeoff in macaque area LIP. eLife 3: e02260, 2014. [PMC free article] [PubMed] [Google Scholar] 200. Hansel D, Sompolinsky H. Synchronization and computation in a chaotic neural network. Phys Rev Lett 68: 718–721, 1992. [PubMed] [Google Scholar] 201. Hariri AR, Mattay VS, Tessitore A, Kolachana B, Fera F, Goldman D, 2 Egan MF, Weinberger DR. Serotonin transporter genetic variation and the response of the human amygdala. Science 297: 400–403, 2002. [PubMed] [Google Scholar] 202. Harlow HF. The formation of learning sets. Psychol Rev 56: 51–65, 1949. [PubMed] [Google Scholar] 203. Harnett MT, Bernier BE, Ahn KC, Morikawa H. Burst-timing-dependent plasticity of NMDA receptor-mediated transmission in midbrain dopamine neurons. Neuron 62: 826–838, 2009. [PMC free article] [PubMed] [Google Scholar] 204. Hart AS, Rutledge RB, Glimcher PW, Phillips PEM. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci 34: 698–704, 2014. [PMC free article] [PubMed] [Google Scholar] 205. Hassani OK, Cromwell HC, Schultz W. Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. J Neurophysiol 85: 2477–2489, 2001. [PubMed] [Google Scholar] 206. Hayden BY, Heilbronner SR, Pearson JM, Platt ML. Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior. J Neurosci 31: 4178–4187, 2011. [PMC free article] [PubMed] [Google Scholar] 207. Hayden BY, Heilbronner SR, Platt ML. Ambiguity aversion in rhesus macaques. Front Neurosci 4: 166: 1–7, 2010. [PMC free article] [PubMed] [Google Scholar] 208. Hayden BY, Pearson JM, Platt ML. Fictive reward signals in the anterior cingulate cortex. Science 324: 948–950, 2009. [PMC free article] [PubMed] [Google Scholar] 209. Hayden BY, Pearson JM, Platt ML. Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci 14: 933–939, 2011. [PMC free article] [PubMed] [Google Scholar] 210. Hayden BY, Platt ML. Neurons in anterior cingulate cortex multiplex information about reward and action. J Neurosci 30: 3339–3346, 2010. [PMC free article] [PubMed] [Google Scholar] 211. Henry DJ, Greene MA, White FJ. Electrophysiological effects of cocaine in the mesoaccumbens dopamine system: Repeated administration. J Pharm Exp Ther 251: 833–839, 1989. [PubMed] [Google Scholar] 212. Hernández A, Zainos A, Romo R. Temporal evolution of a decision-making process in medial premotor cortex. Neuron 33: 959–972, 2002. [PubMed] [Google Scholar] 213. Hernández-López S, Bargas J, Surmeier DJ, Reyes A, Galarraga E. D1 receptor activation enhances evoked discharge in neostriatal medium spiny neurons by modulating an L-type Ca2+ conductance. J Neurosci 17: 3334–3342, 1997. [PMC free article] [PubMed] [Google Scholar] 214. Hernández-López S, Tkatch T, Perez-Garci E, Galarraga E, Bargas J, Hamm H, Surmeier DJ. D2 dopamine receptors in striatal medium spiny neurons reduce L-type Ca2+ currents and excitability via a novel PLCβ1-IP3-calcineurin-signaling cascade. J Neurosci 20, 8987–8995, 2000. [PMC free article] [PubMed] [Google Scholar] 215. Hikosaka O, Sakamoto M, Usui S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J Neurophysiol 61: 814–832, 1989. [PubMed] [Google Scholar] 216. Hikosaka K, Watanabe M. Long-range and short-range reward expectancy in the primate orbitofrontal cortex. Eur J Neurosci 19: 1046–1054, 2004. [PubMed] [Google Scholar] 217. Histed MH, Pasupathy A, Miller EK. Learning substrates in the primate prefrontal cortex and striatum: sustained activity related to successful actions. Neuron 63: 244–253, 2009. [PMC free article] [PubMed] [Google Scholar] 218. Ho MY, Mobini S, Chinang TJ, Bradshaw CM, Szabadi E. Theory and method in the quantitative analysis of “impulsive choice” behaviour: implications for psychopharmacology. Psychopharmacology 146: 362–372, 1999. [PubMed] [Google Scholar] 219. Hofstadter DR. Gödel, Escher, Bach: The Eternal Golden Braid. New York: Basics Books, 1979, p 711. [Google Scholar] 220. Holland PC. CS-US interval as a determinant of the form of Pavlovian appetitive conditioned responses. J Exp Psychol Anim Behav Process 6: 155–174, 1980. [PubMed] [Google Scholar] 221. Hollerman JR, Schultz W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1: 304–309, 1998. [PubMed] [Google Scholar] 222. Hollerman JR, Tremblay L, Schultz W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80: 947–963, 1998. [PubMed] [Google Scholar] 223. Hollis KL, Pharr VL, Dumas MJ, Britton GB, Field J. Classical conditioning provides paternity advantage for territorial male blue gouramis (Trichogaster trichopterus). J Comp Psychol 111: 219–225, 1997. [Google Scholar] 224. Hollon NG, Arnold MM, Gan JO, Walton ME, Phillips PEM. Dopamine-associated cached values are not sufficient as the basis for action selection. Proc Natl Acad Sci USA 111: 18357–18362, 2014. [PMC free article] [PubMed] [Google Scholar] 225. Holt CA, Laury SK. Risk aversion and incentive effects. Am Econ Rev 92: 1644–1655, 2002. [Google Scholar] 226. Hong S, Jhou TC, Smith M, Saleem KS, Hikosaka O. Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J Neurosci 31: 11457–11471, 2011. [PMC free article] [PubMed] [Google Scholar] 227. Hong S, Hikosaka O. The globus pallidus sends reward-related signals to the lateral habenula. Neuron 60, 720–729, 2008. [PMC free article] [PubMed] [Google Scholar] 228. Hong S, Hikosaka O. Pedunculopontine tegmental nucleus neurons provide reward, sensorimotor, and alerting signals to midbrain dopamine neurons. Neuroscience 282: 139–155, 2014. [PMC free article] [PubMed] [Google Scholar] 229. Horvitz JC, Stewart Jacobs BL. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res 759: 251–258, 1997. [PubMed] [Google Scholar] 230. Horwitz GD, Batista AP, Newsome WT. Representation of an abstract perceptual decision in macaque superior colliculus. J Neurophysiol 91: 2281–2296, 2004. [PubMed] [Google Scholar] 231. Hosokawa T, Kato K, Inoue M, Mikami A. Neurons in the orbitofrontal cortex code both visual shapes and reward types. NeuroReport 15: 1493–1496, 2004. [PubMed] [Google Scholar] 232. Hosokawa T, Kato K, Inoue M, Mikami A. Neurons in the macaque orbitofrontal cortex code relative preference of both rewarding and aversive outcomes. Neurosci Res 57: 434–445, 2007. [PubMed] [Google Scholar] 233. Hosokawa T, Kennerley SW, Sloan J, Wallis JD. Single-neuron mechanisms underlying cost-benefit analysis in frontal cortex. J Neurosci 33: 17385–17397, 2013. [PMC free article] [PubMed] [Google Scholar] 234. Hosokawa T, Watanabe M. Prefrontal neurons represent winning and losing during competitive video shooting games between monkeys. J Neurosci 32: 7662–7671, 2012. [PMC free article] [PubMed] [Google Scholar] 235. Hosoya T, Baccus SA, Meister M. Dynamic predictive coding by the retina. Nature 436: 71–77, 2005. [PubMed] [Google Scholar] 236. Houk JC, Adams JL, Barto AG. A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of Information Processing in the Basal Ganglia, edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press, 1995, p. 249–270. [Google Scholar] 237. Howe MW, Tierney PL, Sandberg SG, Phillips PEM, Graybiel AM. Prolonged dopamine signaling in striatum signals proximity and value of distant rewards. Nature 500: 575–579, 2013. [PMC free article] [PubMed] [Google Scholar] 238. Huang CF, Litzenberger RH. Foundations for Financial Economics. Upper Saddle River, NJ: Prentice-Hall, 1988. [Google Scholar] 239. Hrupka BJ, Lin YM, Gietzen DW, Rogers QR. Small changes in essential amino acid concentrations alter diet selection in amino acid-deficient rats. J Nutr 127: 777–784, 1997. [PubMed] [Google Scholar] 240. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. Neural systems responding to degrees of uncertainty in human decision-making. Science 310: 1680–1683, 2005. [PubMed] [Google Scholar] 241. Hsu M, Krajbich I, Zhao C, Camerer CF. Neural response to reward anticipation under risk is nonlinear in probabilities. J Neurosci 29: 2237–2231, 2009. [PMC free article] [PubMed] [Google Scholar] 242. Hull CL. Principles of Behavior. New York: Appleton-Century-Crofts, 1943. [Google Scholar] 243. Ikeda T, Hikosaka O. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39: 693–700, 2003. [PubMed] [Google Scholar] 244. Ilango A, Kesner AJ, Keller KL, Stuber GD, Bonci A, Ikemoto S. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J Neurosci 34: 817–822, 2014. [PMC free article] [PubMed] [Google Scholar] 245. Ito M, Doya K. Validation of decision-making models and analysis of decision variables in the rat basal ganglia. J Neurosci 29: 9861–9874, 2009. [PMC free article] [PubMed] [Google Scholar] 246. Ito S, Stuphorn V, Brown JW, Schall JD. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302: 120–122, 2003. [PubMed] [Google Scholar] 247. Izhikevich EM. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 17: 2443–2452, 2007. [PubMed] [Google Scholar] 248. Janssen P, Shadlen MN. A representation of the hazard rate of elapsed time in macaque area LIP. Nat Neurosci 8: 234–241, 2005. [PubMed] [Google Scholar] 249. Jhou TC, Fields HL, Baxter MB, Saper CB, Holland PC. The rostromedial tegmental nucleus (RMTg), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron 61: 786–800, 2009. [PMC free article] [PubMed] [Google Scholar] 250. Ji H, Shepard PD. Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABAA receptor-mediated mechanism. J Neurosci 27: 6923–6930, 2007. [PMC free article] [PubMed] [Google Scholar] 251. Jimenez-Castellanos J, Graybiel AM. Subdivisions of the dopamine-containing A8-A9-A10 complex identified by their differential mesostriatal innervation of striosomes and extrastriosomal matrix. Neuroscience 23: 223–242, 1987. [PubMed] [Google Scholar] 252. Jimenez-Castellanos J, Graybiel AM. Evidence that histochemically distinct zones of the primate substantia nigra pars compacta are related to patterned distributions of nigrostriatal projection neurons and striatonigral fibers. Exp Brain Res 74: 227–238, 1989. [PubMed] [Google Scholar] 253. Jog MS, Kubota Y, Connolly CI, Hillegaart V, Graybiel AM. Building neural representations of habits. Science 286: 1745–1749, 1999. [PubMed] [Google Scholar] 254. Johnson JG, Busemeyer JR. A dynamic, stochastic, computational model of preference reversal phenomena. Psychol Rev 112: 841–861, 2005. [PubMed] [Google Scholar] 255. Joshua M, Adler A, Mitelman R, Vaadia E, Bergman H. Midbrain dopaminergic neurons and striatal cholinergic interneurons encode the difference between reward and aversive events at different epochs of probabilistic classical conditioning trials. J Neurosci 28: 11673–11684, 2008. [PMC free article] [PubMed] [Google Scholar] 256. Joshua M, Adler A, Prut Y, Vaadia E, Wickens JR, Hagai Bergman H. Synchronization of midbrain dopaminergic neurons is enhanced by rewarding events. Neuron 62: 695–704, 2009. [PubMed] [Google Scholar] 257. Kagel JH, Battalio RC, Green L. Economic Choice Theory: An Experimental Analysis of Animal Behavior. Cambridge, UK: Cambridge Univ. Press, 1995. [Google Scholar] 258. Kahneman D, Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 47: 263–291, 1979. [Google Scholar] 259. Kahneman D, Wakker PP, Sarin R. Back to Bentham? Explorations of experienced utility. Q J Econ 112: 375–405, 1997. [Google Scholar] 260. Kawagoe R, Takikawa Y, Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1: 411–416, 1998. [PubMed] [Google Scholar] 261. Kennerley SW, Behrens TEJ, Wallis JD. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat Neurosci 14: 1581–1589, 2011. [PMC free article] [PubMed] [Google Scholar] 262. Kennerley SW, Wallis JD. Reward-dependent modulation of working memory in lateral prefrontal cortex. J Neurosci 29: 3259–3270, 2009. [PMC free article] [PubMed] [Google Scholar] 263. Kennerley SW, Wallis JD. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci 29: 2061–2073, 2009. [PMC free article] [PubMed] [Google Scholar] 264. Kepecs A, Uchida N, Zariwala H, Mainen ZF. Neural correlates, computation and behavioural impact of decision confidence. Nature 455: 227–231, 2008. [PubMed] [Google Scholar] 265. Kerr JN, Wickens JR. Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. J Neurophysiol 85: 117–124, 2001. [PubMed] [Google Scholar] 266. Kettner RE, Mahamud S, Leung HC, Sitkoff N, Houk JC, Peterson BW, Barto AG. Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movements. J Neurophysiol 77: 2115–2130, 1997. [PubMed] [Google Scholar] 267. Khamassi M, Quilodran R, Pierre Enel P, Dominey PF, Procyk E. Behavioral regulation and the modulation of information coding in the lateral prefrontal and cingulate cortex. Cereb Cortex. In press. [PubMed] [Google Scholar] 268. Kheramin S, Body S, Ho MY, Velazquez-Martinez DN, Bradshaw CM, Szabadi E, Deakin JFW, anderson IM. Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: a quantitative analysis. Psychopharmacology 175: 206–214, 2004. [PubMed] [Google Scholar] 269. Kiani R, Cueva CJ, Reppas JB, Newsome WT. Dynamics of neural population responses in prefrontal cortex indicate changes of mind on single trials. Curr Biol 24: 1542–1547, 2014. [PMC free article] [PubMed] [Google Scholar] 270. Kiani R, Hanks TD, Shadlen MN. Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. J Neurosci 28: 3017–3029, 2008. [PMC free article] [PubMed] [Google Scholar] 271. Kim KM, Baratta MV, Yang A, Lee D, Boyden ES, Fiorillo CD. Optogenetic mimicry of the transient activation of dopamine neurons by natural reward is sufficient for operant reinforcement. PLoS One 7: e33612, 2012. [PMC free article] [PubMed] [Google Scholar] 272. Kim S, Hwang J, Lee D. Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59: 161–172, 2008. [PMC free article] [PubMed] [Google Scholar] 273. Kim JJ, Krupa DJ, Thompson RF. Inhibitory cerebello-olivary projections and blocking effect in classical conditioning. Science 279: 570–573, 1998. [PubMed] [Google Scholar] 274. Kim JN, Shadlen MN. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2: 176–185, 1999. [PubMed] [Google Scholar] 275. Kim H, Sul JH, Huh N, Lee D, Jung MW. Role of striatum in updating values of chosen actions. J Neurosci 29: 14701–14712, 2009. [PMC free article] [PubMed] [Google Scholar] 276. Kimura M, Rajkowski J, Evarts E. Tonically discharging putamen neurons exhibit set-dependent responses. Proc Natl Acad Sci USA 81: 4998–5001, 1984. [PMC free article] [PubMed] [Google Scholar] 277. Kirby KN, Marakovic NN. Delay-discounting probabilistic rewards: rates decrease as amounts increase. Psychonom Bull Rev 3: 100–104, 1996. [PubMed] [Google Scholar] 278. Kitazawa S, Kimura T, Yin PB. Cerebellar complex spikes encode both destinations and errors in arm movement. Nature 392: 494–497, 1998. [PubMed] [Google Scholar] 279. Klein JT, Deaner RO, Platt ML. Neural correlates of social target value in macaque parietal cortex. Curr Biol 18: 419–424, 2008. [PMC free article] [PubMed] [Google Scholar] 280. Knutson B, Adams CM, Fong GW, Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21: 1–5, 2001. [PMC free article] [PubMed] [Google Scholar] 281. Kobayashi Y, Inoue Y, Yamamoto M, Isa T, Aizawa H. Contribution of pedunculopontine tegmental nucleus neurons to performance of visually guided saccade tasks in monkeys. J Neurophysiol 88: 715–731, 2002. [PubMed] [Google Scholar] 282. Kobayashi S, Lauwereyns J, Koizumi M, Sakagami M, Hikosaka O. Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J Neurophysiol 87: 1488–1498, 2002. [PubMed] [Google Scholar] 283. Kobayashi S, Nomoto K, Watanabe M, Hikosaka O, Schultz W, Sakagami M. Influences of rewarding and aversive outcomes on activity in macaque lateral prefrontal cortex. Neuron 51: 861–870, 2006. [PubMed] [Google Scholar] 284. Kobayashi S, Pinto de Carvalho O, Schultz W. Adaptation of reward sensitivity in orbitofrontal neurons. J Neurosci 30: 534–544, 2010. [PMC free article] [PubMed] [Google Scholar] 285. Kobayashi S, Schultz W. Influence of reward delays on responses of dopamine neurons. J Neurosci 28: 7837–7846, 2008. [PMC free article] [PubMed] [Google Scholar] 286. Kobayashi S, Schultz W. Reward contexts extend dopamine signals to unrewarded stimuli. Curr Biol 24: 56–62, 2014. [PMC free article] [PubMed] [Google Scholar] 287. Koch C. Biophysics of Computation. New York: Oxford Univ. Press, 1999. [Google Scholar] 288. Komura Y, Nikkuni A, Hirashima N, Uetake T, Miyamoto A. Responses of pulvinar neurons reflect a subject's confidence in visual categorization. Nat Neurosci 16: 749–755, 2013. [PubMed] [Google Scholar] 289. Konorski J. Integrative Activity of the bBrain. Chicago: Univ. of Chicago Press, 1967. [Google Scholar] 290. Köszegi B, Rabin M. A model of reference-dependent preferences. Q J Econ 121: 1133–1165, 2006. [Google Scholar] 291. Krajbich I, Armel C, Rangel R. Visual fixations and the computation and comparison of value in simple choice. Nat Neurosci 13: 1292–1298, 2010. [PubMed] [Google Scholar] 292. Krauzlis RJ, Basso MA, Wurtz RH. Shared motor error for multiple eye movements. Science 276: 1693–1695, 1997. [PubMed] [Google Scholar] 293. Kravitz AV, Tye LD, Kreitzer AC. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci 15: 816–818, 2012. [PMC free article] [PubMed] [Google Scholar] 294. Kreitzer AC, Malenka RC. Endocannabinoid-mediated rescue of striatal LTD and motor deficits in Parkinson's disease models. Nature 445: 643–647, 2007. [PubMed] [Google Scholar] 295. Kreps DM. A Course in Microeconomic Theory. Harlow: Pearson Education, 1990. [Google Scholar] 296. Kreps DM, Porteus E. Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46: 185–200, 1978. [Google Scholar] 297. Kuhn A, Aertsen A, Rotter S. Neuronal integration of synaptic input in the fluctuation-driven regime. J Neurosci 24, 2345–2356, 2004. [PMC free article] [PubMed] [Google Scholar] 298. Kurata K, Wise SP. Premotor cortex of rhesus monkeys: set-related activity during two conditional motor tasks. Exp Brain Res 69: 327–343, 1988. [PubMed] [Google Scholar] 299. Kurata K, Wise SP. Premotor and supplementary motor cortex in rhesus monkeys: neuronal activity during externally- and internally-instructed motor tasks. Exp Brain Res 72: 237–248, 1988. [PubMed] [Google Scholar] 300. Lak A, Arabzadeh E, Harris JA, Diamond ME. Correlated physiological and perceptual effects of noise in a tactile stimulus. Proc Natl Acad Sci USA 107: 7981–7986, 2010. [PMC free article] [PubMed] [Google Scholar] 301. Lak A, Stauffer WR, Schultz W. Dopamine prediction error responses integrate subjective value from different reward dimensions. Proc Natl Acad Sci USA 111: 2343–2348, 2014. [PMC free article] [PubMed] [Google Scholar] 302. Laming DRJ. Information Theory of Choice Reaction Time. New York: Wiley, 1968. [Google Scholar] 303. Lammel S, Lim BK, Ran C, Huang KW, Betley MJ, Tye KM, Deisseroth K, Malenka RC. Input-specific control of reward and aversion in the ventral tegmental area. Nature 491: 212–217, 2012. [PMC free article] [PubMed] [Google Scholar] 304. Lammel S, Steinberg E, Földy C, Wall NR, Beier K, Luo L, Malenka RC. Diversity of transgenic mouse models for selective Ttrgeting of midbrain dopamine neurons. Neuron 85: 429–438, 2015. [PMC free article] [PubMed] [Google Scholar] 305. Lattimore PK, Baker JR, Witte AD. The influence of probability on risky choice: a parametric examination. J Econ Behav Organ 17: 377–400, 1992. [Google Scholar] 306. Lau B, Glimcher PW. Value representations in the primate striatum during matching behavior. Neuron 58: 451–463, 2008. [PMC free article] [PubMed] [Google Scholar] 307. Laughlin S. A simple coding procedure enhances a neuron's information capacity. Z Naturforsch 36: 910–912, 1981. [PubMed] [Google Scholar] 308. Lauwereyns J, Takikawa Y, Kawagoe R, Kobayashi S, Koizumi M, Coe B, Sakagami M, Hikosaka O. Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron 33: 463–473, 2002. [PubMed] [Google Scholar] 309. Leathers ML, Olson CR. In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science 338: 132–135, 2012. [PMC free article] [PubMed] [Google Scholar] 310. Lecourtier L, DeFrancesco A, Moghaddam B. Differential tonic influence of lateral habenula on prefrontal cortex and nucleus accumbens dopamine release. Eur J Neurosci 27: 1755–1762, 2008. [PMC free article] [PubMed] [Google Scholar] 311. Lee IH, Assad JA. Putaminal activity for simple reactions or self-timed movements. J Neurophysiol 89: 2528–2537, 2003. [PubMed] [Google Scholar] 312. Lemus L, Hernández A, Luna R, Zainos A, Nácher V, Romo R. Neural correlates of a postponed decision report. Proc Natl Acad Sci USA 104: 17174–17179, 2007. [PMC free article] [PubMed] [Google Scholar] 313. Leon MI, Shadlen MN. Effect of expected reward magnitude on the responses of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24: 415–425, 1999. [PubMed] [Google Scholar] 314. Leung PMB, Rogers QR. Importance of prepyriform cortex in food-intake response of rats to amino acids. Am J Physiol 221: 929–935, 1971. [PubMed] [Google Scholar] 315. Levy H, Markowitz HM. Approximating expected utility by a function of mean and variance. Am Econ Rev 69: 308–317, 1979. [Google Scholar] 316. Libet B, Gleason CA, Wright EW, Pearl DK. Time of conscious intention to act in relation to onset of cerebral activities (readiness-potential): the unconscious initiation of a freely voluntary act. Brain 106: 623–642, 1983. [PubMed] [Google Scholar] 317. Lishman WA. Organic Psychiatry. Oxford: Blackwell, 1998. [Google Scholar] 318. Liu QS, Pu L, Poo MM. Repeated cocaine exposure in vivo facilitates LTP induction in midbrain dopamine neurons. Nature 437: 1027–1031, 2005. [PMC free article] [PubMed] [Google Scholar] 319. Liu Z, Richmond BJ. Response differences in monkey TE and perirhinal cortex: stimulus association related to reward schedules. J Neurophysiol 83: 1677–1692, 2000. [PubMed] [Google Scholar] 320. Livio M. The Golden Ratio: The Story of PHI, the World's Most Astonishing Number. New York: Random House, 2002. [Google Scholar] 321. Ljungberg T, Apicella P, Schultz W. Responses of monkey midbrain dopamine neurons during delayed alternation performance. Brain Res 586: 337–341, 1991. [PubMed] [Google Scholar] 322. Ljungberg T, Apicella P, Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J Neurophysiol 67: 145–163, 1992. [PubMed] [Google Scholar] 323. Lo CC, Wang XJ. Cortico-basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat Neurosci 9: 956–963, 2006. [PubMed] [Google Scholar] 324. Loewenstein G, Thompson L, Bazerman MH. Social utility and decision making in interpersonal contexts. J Personality Soc Psych 57: 426–441, 1989. [Google Scholar] 325. Loewenstein G, Prelec D. Anomalies in intertemporal choice: evidence and an interpretation. Q J Econ 107: 573–597, 1992. [Google Scholar] 326. Leszczuk MH, Flaherty CF. Lesions of nucleus accumbens reduce instrumental but not consummatory negative contrast in rats. Behav Brain Res 116: 61–79, 2000. [PubMed] [Google Scholar] 327. Logan GD, Cowan WB. On the ability to inhibit thought and action: a theory of an act of control. Psychol Rev 91: 295–327, 1984. [PubMed] [Google Scholar] 328. Logothetis NK, Pauls J, Augath M, Trinath T, Oeltermann A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412: 150–157, 2001. [PubMed] [Google Scholar] 329. Lohrenz T, McCabe K, Camerer CF, Montague PR. Neural signature of fictive learning signals in a sequential investment task. Proc Natl Acad Sci USA 104: 9493–9498, 2007. [PMC free article] [PubMed] [Google Scholar] 330. Longtin A. Stochastic resonance in neuron models. J Stat Phys 70: 309, 1993. [Google Scholar] 331. Louie K, Glimcher PW. Separating value from choice: delay discounting activity in the lateral intraparietal area. J Neurosci 30: 5498–5507, 2010. [PMC free article] [PubMed] [Google Scholar] 332. Louie K, Grattan LE, Glimcher PW. Reward value-based gain control: divisive normalization in parietal cortex. J Neurosci 31: 10627–10639, 2011. [PMC free article] [PubMed] [Google Scholar] 333. Luce RD. Individual Choice Behavior: A Theoretical Analysis. New York: Wiley, 1959. [Google Scholar] 334. Machens CK, Romo R, Brody CD. Flexible control of mutual inhibition: a neural model of two-interval discrimination. Science 307: 1121–1124, 2005. [PubMed] [Google Scholar] 335. Mackintosh NJ. The Psychology of Animal Learning. London: Academic Press, 1974. [Google Scholar] 336. Mackintosh NJ. A theory of attention: variations in the associability of stimulus with reinforcement. Psychol Rev 82: 276–298, 1975. [Google Scholar] 337. Maimon G, Assad JA. Parietal area 5 and the initiation of self-timed movements versus simple reactions. J Neurosci 26: 2487–2498, 2006. [PMC free article] [PubMed] [Google Scholar] 338. Markowitsch HJ, Pritzel M. Reward-related neurons in cat association cortex. Brain Res 111: 185–188, 1976. [PubMed] [Google Scholar] 339. Markowitz H. The utility of wealth. J Polit Econ 6: 151–158, 1952. [Google Scholar] 340. Martínez-García M, Rolls ET, Deco G, Romo R. Neural and computational mechanisms of postponed decisions. Proc Natl Acad Sci 108: 11626–11631, 2011. [PMC free article] [PubMed] [Google Scholar] 341. Mas-Colell A, Whinston M, Green J. Microeconomic Theory. New York: Oxford Univ. Press, 1995. [Google Scholar] 342. Matsuda W, Furuta T, Nakamura KC, Hioki H, Fujiyama F, Arai R, Kaneko T. Single nigrostriatal dopaminergic neurons form widely spread and highly dense axonal arborizations in the neostriatum. J Neurosci 29: 444–453, 2009. [PMC free article] [PubMed] [Google Scholar] 343. Matsuda Y, Marzo A, Otani S. The presence of background dopamine signal converts long-term synaptic depression to potentiation in rat prefrontal cortex. J Neurosci 26: 4803–4810, 2006. [PMC free article] [PubMed] [Google Scholar] 344. Matsumoto M, Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447: 1111–1115, 2007. [PubMed] [Google Scholar] 345. Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctively convey positive and negative motivational signals. Nature 459: 837–841, 2009. [PMC free article] [PubMed] [Google Scholar] 346. Matsumoto M, Hikosaka O. Representation of negative motivational value in the primate lateral habenula. Nat Neurosci 12: 77–84, 2009. [PMC free article] [PubMed] [Google Scholar] 347. Matsumoto M, Matsumoto K, Abe H, Tanaka K. Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci 10: 647–656, 2007. [PubMed] [Google Scholar] 348. Matsumoto K, Suzuki W, Tanaka K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301: 229–232, 2003. [PubMed] [Google Scholar] 349. Matsumoto M, Takada M. Distinct representations of cognitive and motivational signals in midbrain dopamine neurons. Neuron 79: 1011–1024, 2013. [PubMed] [Google Scholar] 350. Mauritz KH, Wise SP. Premotor cortex of the rhesus monkey: neuronal activity in anticipation of predictable environmental events. Exp Brain Res 61: 229–244, 1986. [PubMed] [Google Scholar] 351. McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339–346, 2003. [PubMed] [Google Scholar] 352. McClure SM, Laibson DI, Loewenstein G, Cohen JD. Separate neural systems value immediate and delayed monetary rewards. Science 306: 503–507, 2004. [PubMed] [Google Scholar] 353. McClure SM, Li J, Tomlin D, Cypert KS, Montague LM, Montague PR. Neural correlates of behavioral preference for culturally familiar drinks. Neuron 44: 379–387, 2004. [PubMed] [Google Scholar] 354. McCoy AN, Crowley JC, Haghighian G, Dean HL, Platt ML. Saccade reward signals in posterior cingulate cortex. Neuron 40: 1031–1040, 2003. [PubMed] [Google Scholar] 355. McCoy AN, Platt ML. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 8: 1220–1227, 2005. [PubMed] [Google Scholar] 356. McEchron MD, Bouwmeester H, Tseng W, Weiss C, Disterhoft JF. Hippocampectomy disrupts auditory trace fear conditioning and contextual fear conditioning in the rat. Hippocampus 8: 638–646, 1998. [PubMed] [Google Scholar] 357. Medina JF, Nores WL, Mauk MD. Inhibition of climbing fibers is a signal for the extinction of conditioned eyelid responses. Nature 416: 330–333, 2002. [PubMed] [Google Scholar] 358. Meijer JH, Robbers Y. Wheel running in the wild. Proc R Soc B 281: 2014.0210, 2014. [PMC free article] [PubMed] [Google Scholar] 359. Melis AP, Hare B, Tomasello M. Engineering cooperation in chimpanzees: tolerance constraints on cooperation. Anim Behav 72: 275–286, 2006. [Google Scholar] 360. Merten K, Nieder A. Active encoding of decisions about stimulus absence in primate prefrontal cortex neurons. Proc Natl Acad Sci USA 109: 6289–6294, 2012. [PMC free article] [PubMed] [Google Scholar] 361. Merten K, Nieder A. Comparison of abstract decision encoding in the monkey prefrontal cortex, the presupplementary, and cingulate motor areas. J Neurophysiol 110: 19–32, 2013. [PubMed] [Google Scholar] 362. Mileykovskiy B, Morales M. Duration of inhibition of ventral tegmental area dopamine neurons encodes a level of conditioned fear. J Neurosci 31: 7471–7476, 2011. [PMC free article] [PubMed] [Google Scholar] 363. Mill JS. Utilitarianism, London: Parker, Son and Bourn, 1863. [Google Scholar] 364. Miller LA. Cognitive risk-taking after frontal or temporal lobectomy. I: Synthesis of fragmented visual information. Neuropsychologia 23: 359–369, 1985. [PubMed] [Google Scholar] 365. Mirenowicz J, Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J Neurophysiol 72: 1024–1027, 1994. [PubMed] [Google Scholar] 366. Mirenowicz J, Schultz W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379: 449–451, 1996. [PubMed] [Google Scholar] 367. Mitchell DS, Gormezano I. Effects of water deprivation on classical appetitive conditioning of the rabbit's jaw movement response. Learn Motivat 1: 199–206, 1970. [Google Scholar] 368. Mink JW. The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50: 381–425, 1996. [PubMed] [Google Scholar] 369. Mizuhiki T, Richmond BJ, Shidara M. Encoding of reward expectation by monkey anterior insular neurons. J Neurophysiol 107: 2996–3007, 2012. [PMC free article] [PubMed] [Google Scholar] 370. Mobini S, Body S, Ho MY, Bradshaw CM, Szabadi E, Deakin JFW, anderson IM. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology 160: 290–298, 2002. [PubMed] [Google Scholar] 371. Mogami T, Tanaka K. Reward association affects neuronal responses to visual stimuli in macaque TE and perirhinal cortices. J Neurosci 26: 6761–6770, 2006. [PMC free article] [PubMed] [Google Scholar] 372. Monosov IE, Hikosaka O. Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex. J Neurosci 32: 10318–10330, 2012. [PMC free article] [PubMed] [Google Scholar] 373. Monosov IE, Hikosaka O. Selective and graded coding of reward uncertainty by neurons in the primate anterodorsal septal region. Nat Neurosci 16: 756–762, 2013. [PMC free article] [PubMed] [Google Scholar] 374. Montague PR, Dayan P, Sejnowski TJ. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16: 1936–1947, 1996. [PMC free article] [PubMed] [Google Scholar] 375. Montague PR, Sejnowski TJ. The predictive brain: temporal coincidence and temporal order in synaptic learning mechanisms. Learn Mem 1: 1–33, 1994. [PubMed] [Google Scholar] 376. Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 43: 133–143, 2004. [PubMed] [Google Scholar] 377. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H. Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9: 1057–1063, 2006. [PubMed] [Google Scholar] 378. Morrison SE, Saez A, Lau B, Salzman CD. Different time courses for learning-related changes in amygdala and orbitofrontal cortex. Neuron 71, 1127–1140, 2011. [PMC free article] [PubMed] [Google Scholar] 379. Munoz DP, Wurtz RH. Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. J Neurophysiol 73: 2313–2333, 1995. [PubMed] [Google Scholar] 380. Murakami M, Vicente MI, Costa GM, Mainen ZF. Neuronal antecedents of self-initiated actions in secondary motor cortex. Nat Neurosci 17: 1574–1582, 2014. [PubMed] [Google Scholar] 381. Musallam S, Corneil BD, Greger B, Scherberger H, andersen RA. Cognitive control signals for neural prosthetics. Science 305: 258–262, 2004. [PubMed] [Google Scholar] 382. Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O. Dopamine neurons can represent context-dependent prediction error. Neuron 41: 269–280, 2004. [PubMed] [Google Scholar] 383. Nakamura K, Mikami A, Kubota K. Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. J Neurophysiol 67: 1447–1463, 1992. [PubMed] [Google Scholar] 384. Nassar MR, Wilson RC, Heasly B, Gold JI. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J Neurosci 30: 12366–12378, 2010. [PMC free article] [PubMed] [Google Scholar] 385. Newsome WT, Britten KH, Movshon JA. Neuronal correlates of a perceptual decision. Nature 341: 52–54, 1989. [PubMed] [Google Scholar] 386. Niki H, Watanabe M. Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res 171: 213–224, 1979. [PubMed] [Google Scholar] 387. Nishijo H, Ono T, Nishino H. Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance. J Neurosci 8: 3570–3583, 1988. [PMC free article] [PubMed] [Google Scholar] 388. Niv Y, Duff MO, Dayan P. Dopamine, uncertainty and TD learning. Behav Brain Func 1:6, 2005. [PMC free article] [PubMed] [Google Scholar] 389. Nomoto K, Schultz W, Watanabe T, Sakagami M. Temporally extended dopamine responses to perceptually demanding reward-predictive stimuli. J Neurosci 30: 10692–10702, 2010. [PMC free article] [PubMed] [Google Scholar] 390. O'Doherty J, Dayan P, Friston K, Critchley H, Dolan RJ. Temporal difference models and reward-related lNiv earning in the human brain. Neuron 28: 329–337, 2003. [PubMed] [Google Scholar] 391. O'Neill M, Schultz W. Coding of reward risk distinct from reward value by orbitofrontal neurons. Neuron 68: 789–800, 2010. [PubMed] [Google Scholar] 392. O'Neill M, Schultz W. Risk prediction error coding in orbitofrontal neurons. J Neurosci 33: 15810–15814, 2013. [PMC free article] [PubMed] [Google Scholar] 393. Ogawa M, van der Meer MAA, Esber GR, Cerri DH, Stalnaker TA, Schoenbaum G. Risk-responsive orbitofrontal neurons track acquired salience. Neuron 77: 251–258, 2013. [PMC free article] [PubMed] [Google Scholar] 394. Ohyama K, Sugase-Miyamoto Y, Matsumoto N, Shidara M, Sato C. Stimulus-related activity during conditional associations in monkey perirhinal cortex neurons depends on upcoming reward outcome. J Neurosci 32: 17407–17419, 2012. [PMC free article] [PubMed] [Google Scholar] 395. Ojakangas CL, Ebner TJ. Purkinje cell complex and simple spike changes during a voluntary arm movement learning task in the monkey. J Neurophysiol 68: 2222–2236, 1992. [PubMed] [Google Scholar] 396. Okada KI, Toyama K, Inoue Y, Isa T, Kobayashi Y. Different pedunculopontine tegmental neurons signal predicted and actual task rewards. J Neurosci 29: 4858–4870, 2009. [PMC free article] [PubMed] [Google Scholar] 397. Okano K, Tanji J. Neuronal activities in the primate motor fields of the agranular frontal cortex preceding visually triggered and self-paced movement. Exp Brain Res 66: 155–166, 1987. [PubMed] [Google Scholar] 398. Olds J, Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol 47: 419–427, 1954. [PubMed] [Google Scholar] 399. Ostlund SB, Balleine BW. Differential involvement of the basolateral amygdala and mediodorsal thalamus in instrumental action selection. J Neurosci 28: 4398–4405, 2008. [PMC free article] [PubMed] [Google Scholar] 400. Otani S, Blond O, Desce JM, Crepel F. Dopamine facilitates long-term depression of glutamatergic transmission in rat prefrontal cortex. Neuroscience 85: 669–676, 1998. [PubMed] [Google Scholar] 401. Otmakhova NA, Lisman JE. D1/D5 dopamine receptor activation increases the magnitude of early long-term potentiation at CA1 hippocampal synapses. J Neurosci 16: 7478–7486, 1996. [PMC free article] [PubMed] [Google Scholar] 402. Oyama K, Hernádi I, Iijima T, Tsutsui KI. Reward prediction error coding in dorsal striatal neurons. J Neurosci 30: 11447–11457, 2010. [PMC free article] [PubMed] [Google Scholar] 403. Padoa-Schioppa C. Range-adapting representation of economic value in the orbitofrontal cortex. J Neurosci 29: 14004–14014, 2009. [PMC free article] [PubMed] [Google Scholar] 404. Padoa-Schioppa C. Neuronal origins of choice variability in economic decisions. Neuron 80: 1322–1336, 2013. [PMC free article] [PubMed] [Google Scholar] 405. Padoa-Schioppa C, Assad JA. Neurons in the orbitofrontal cortex encode economic value. Nature 441: 223–226, 2006. [PMC free article] [PubMed] [Google Scholar] 406. Padoa-Schioppa C, Assad JA. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat Neurosci 11: 95–102, 2008. [PMC free article] [PubMed] [Google Scholar] 407. Paladini CA, Celada P, Tepper JM. Striatal, pallidal, and pars reticulata evoked inhibition of nigrostriatal dopaminergic neurons is mediated by GABAa receptors in vivo. Neuroscience 89: 799–812, 1998. [PubMed] [Google Scholar] 408. Pan WX, Brown J, Dudman JT. Neural signals of extinction in the inhibitory microcircuit of the ventral midbrain. Nat Neurosci 16: 71–78, 2013. [PMC free article] [PubMed] [Google Scholar] 409. Pan X, Fan H, Sawa K, Tsuda I, Tsukada M, Sakagami M. Reward inference by primate prefrontal and striatal neurons. J Neurosci 34: 1380–1396, 2014. [PMC free article] [PubMed] [Google Scholar] 410. Pan WX, Hyland BI. Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J Neurosci 25: 4725–4732, 2005. [PMC free article] [PubMed] [Google Scholar] 411. Pan X, Sawa K, Tsuda I, Tsukada M, Sakagami M. Reward prediction based on stimulus categorization in primate lateral prefrontal cortex. Nat Neurosci 11: 703–712, 2008. [PubMed] [Google Scholar] 412. Pan WX, Schmidt R, Wickens JR, Hyland BI. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25: 6235–6242, 2005. [PMC free article] [PubMed] [Google Scholar] 413. Parker JG, Beutler LR, Palmiter RD. The contribution of NMDA receptor signaling in the corticobasal ganglia reward network to appetitive Pavlovian learning. J Neurosci 31: 11362–11369, 2011. [PMC free article] [PubMed] [Google Scholar] 414. Parker JG, Wanat MJ, Soden ME, Ahmad K, Zweifel LS, Bamford NS, Palmiter RD. Attenuating GABAA receptor signaling in dopamine neurons selectively enhances reward learning and alters risk preference in mice. J Neurosci 31: 17103–17112, 2011. [PMC free article] [PubMed] [Google Scholar] 415. Parker JG, Zweifel LS, Clark JJ, Evans SB, Phillips PEM, Palmiter RD. Absence of NMDA receptors in dopamine neurons attenuates dopamine release but not conditioned approach during Pavlovian conditioning. Proc Nat Acad Sci USA 107: 13491–13496, 2010. [PMC free article] [PubMed] [Google Scholar] 416. Parthasarathy HB, Schall JD, Graybiel AM. Distributed but convergent ordering of corticostriatal projections: analysis of the frontal eye field and the supplementary eye field in the macaque monkey. J Neurosci 12: 4468–4488, 1992. [PMC free article] [PubMed] [Google Scholar] 417. Pascal B. Pensées 1658–1662, translated by Ariew R. Indianapolis: Hackett, 2004. [Google Scholar] 418. Pasquereau B, Nadjar A, Arkadir D, Bezard E, Goillandeau M, Bioulac B, Gross CE, Boraud T. Shaping of motor responses by incentive values through the basal ganglia. J Neurosci 27: 1176–1183, 2007. [PMC free article] [PubMed] [Google Scholar] 419. Pasquereau B, Turner RS. Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci 33: 8288–8300, 2013. [PMC free article] [PubMed] [Google Scholar] 420. Pastor-Bernier A, Cisek P. Neural correlates of biased competition in premotor cortex. J Neurosci 31: 7083–7088, 2011. [PMC free article] [PubMed] [Google Scholar] 421. Pasupathy A, Miller EK. Different time courses of learning-related activity in the prefrontal cortex and striaum. Nature 433: 873–876, 2005. [PubMed] [Google Scholar] 422. Paton JJ, Belova MA, Morrison SE, Salzman CD. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439: 865–870, 2006. [PMC free article] [PubMed] [Google Scholar] 423. Pavlov PI. Conditioned Reflexes. London: Oxford Univ. Press, 1927. [Google Scholar] 424. Pawlak V, Kerr JND. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J Neurosci 28: 2435–2446, 2008. [PMC free article] [PubMed] [Google Scholar] 425. Pearce JM, Hall G. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87: 532–552, 1980. [PubMed] [Google Scholar] 426. Pearson JM, PLatt ML. Dopamine: burning the candle at both ends. Neuron 79: 831–833, 2013. [PMC free article] [PubMed] [Google Scholar] 427. Peck CJ, Jangraw DC, Suzuki M, Efem R, Gottlieb J. Reward modulates attention independently of action value in posterior parietal cortex. J Neurosci 29: 11182–11191, 2009. [PMC free article] [PubMed] [Google Scholar] 428. Peck CJ, Lau B, Salzman CD. The primate amygdala combines information about space and value. Nat Neurosci 16: 340–348, 2013. [PMC free article] [PubMed] [Google Scholar] 429. Penrose R. The Emperor's New Mind. Oxford, UK: Oxford Univ. Press, 1989. [Google Scholar] 430. Percheron G, Yelnik J, Francois C. A Golgi analysis of the primate globus pallidus. III. Spatial organization of the striopallidal complex. J Comp Neurol 227: 214–227, 1984. [PubMed] [Google Scholar] 431. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD. Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442: 1042–1045, 2006. [PMC free article] [PubMed] [Google Scholar] 432. Plassmann H, O'Doherty J, Shiv B, Rangel A. Marketing actions can modulate neural representations of experienced pleasantness. Proc Natl Acad Sci USA 105: 1050–1054, 2009. [PMC free article] [PubMed] [Google Scholar] 433. Platt ML, Glimcher PW. Neural correlates of decision variables in parietal cortex. Nature 400: 233–238, 1999. [PubMed] [Google Scholar] 434. Pooresmaeili A, Poort J, Roelfsema PR. Simultaneous selection by object-based attention in visual and frontal cortex. Proc Natl Acad Sci USA 111: 6467–6472, 2014. [PMC free article] [PubMed] [Google Scholar] 435. Prelec D. The probability weighting function. Econometrica 66: 497–527, 1998. [Google Scholar] 436. Prelec D, Loewenstein G. Decision making over time and under uncertainty: a common approach. Management Sci 37: 770–786, 1991. [Google Scholar] 437. Preuschoff K, Bossaerts P, Quartz SR. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 51: 381–390, 2006. [PubMed] [Google Scholar] 438. Preuschoff Bossaerts P. Adding prediction risk to the theory of reward learning. Ann NY Acad Sci 1104: 135–146, 2007. [PubMed] [Google Scholar] 439. Prévost C, Pessiglione M, Météreau E, Cléry-Melin ML, Dreher JC. Separate valuation subsystems for delay and effort decision costs. J Neurosci 30: 14080–14090, 2010. [PMC free article] [PubMed] [Google Scholar] 440. Puig MV, Miller EK. The role of prefrontal dopamine D1 receptors in the neural mechanisms of associative learning. Neuron 74: 874–886, 2012. [PMC free article] [PubMed] [Google Scholar] 441. Purcell BA, Heitz RP, Cohen JY, Schall JD, Logan GD, Palmeri TJ. Neurally constrained modeling of perceptual decision making. Psychol Rev 117: 1113–1143, 2010. [PMC free article] [PubMed] [Google Scholar] 442. Rabin M. Incorporating fairness into game theory and economics. Am Econ Rev 83: 1281–1302, 1993. [Google Scholar] 443. Raby CR, Alexis DM, Dickinson A, Clayton NS. Planning for the future by western scrub-jays. Nature 445: 919–921, 2007. [PubMed] [Google Scholar] 444. Raghuraman AP, Padoa-Schioppa C. Integration of multiple determinants in the neuronal computation of economic values. J Neurosci 34: 11583–11603, 2014. [PMC free article] [PubMed] [Google Scholar] 445. Rahman S, Sahakian BJ, Hodges JR, Rogers RD, Robbins TW. Specific cognitive deficits in mild frontal variant frontotemporal dementia. Brain 122: 1469–1493, 1999. [PubMed] [Google Scholar] 446. Rao RPN, Sejnowski TJ. Self-organizing neural systems based on predictive learning. Phil Trans R Soc A 361: 1149–1175, 2003. [PubMed] [Google Scholar] 447. Ratcliff R. A theory of memory retrieval. Psychol Rev 83: 59–108, 1978. [Google Scholar] 448. Ratcliff R, Cherian A, Segraves M. A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of two-choice decisions. J Neurophysiol 90: 1392–1407, 2003. [PubMed] [Google Scholar] 449. Ratcliff R, Frank MJ. Reinforcement-based decision making in corticostriatal circuits: mutual constraints by neurocomputational and diffusion models. Neural Comput 24: 1186–1229, 2012. [PubMed] [Google Scholar] 450. Ratcliff R, Hasegawa YT, Hasegawa RP, Smith PL, Segraves MA. Dual diffusion model for single-cell recording data from the superior colliculus in a brightness-discrimination task. J Neurophysiol 97: 1756–1774, 2007. [PMC free article] [PubMed] [Google Scholar] 451. Ravel S, Legallet E, Apicella P. Tonically active neurons in the monkey striatum do not preferentially respond to appetitive stimuli. Exp Brain Res 128: 531–534, 1999. [PubMed] [Google Scholar] 452. Ravel S, Legallet E, Apicella P. Responses of tonically active neurons in the monkey striatum discriminate between motivationally opposing stimuli. J Neurosci 23: 8489–8497, 2003. [PMC free article] [PubMed] [Google Scholar] 453. Ravel S, Richmond BJ. Dopamine neuronal responses in monkeys performing visually cued reward schedules. Eur J Neurosci 24: 277–290, 2006. [PubMed] [Google Scholar] 454. Reddi BA, Asrress KN, Carpenter RH. Accuracy, information, and response time in a saccadic decision task. J Neurophysiol 90: 3538–3546, 2003. [PubMed] [Google Scholar] 455. Redgrave P, Prescott TJ, Gurney K. Is the short-latency dopamine response too short to signal reward? Trends Neurosci 22: 146–151, 1999. [PubMed] [Google Scholar] 456. Redgrave P, Gurney K. The short-latency dopamine signal: a role in discovering novel actions? Nat Rev Neurosci 7: 967–975, 2006. [PubMed] [Google Scholar] 457. Redish AD. Addiction as a computational process gone awry. Science 306: 1944–1947, 2004. [PubMed] [Google Scholar] 458. Reed P, Mitchell C, Nokes T. Intrinsic reinforcing properties of putatively neutral stimuli in an instrumental two-lever discrimination task. Anim Learn Behav 24: 38–45, 1996. [Google Scholar] 459. Rescorla RA. Pavlovian conditioning and its proper control procedures. Psychol Rev 74: 71–80, 1967. [PubMed] [Google Scholar] 460. Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II: Current Research and Theory, edited by Black AH, Prokasy WF. New York: Appleton Century Crofts, 1972, p. 64–99. [Google Scholar] 461. Reynolds JNJ, Hyland BI, Wickens JR. A cellular mechanism of reward-related learning. Nature 413: 67–70, 2001. [PubMed] [Google Scholar] 462. Richards JB, Mitchell SH, de Wit H, Seiden LS. Determination of discount functions in rats with an adjusting-amount procedure. J Exp Anal Behav 67: 353–366, 1997. [PMC free article] [PubMed] [Google Scholar] 463. Richards CD, Shiroyama T, Kitai ST. Electrophysiological and immunocytochemical characterization of GABA and dopamine neurons in the substantia nigra of the rat. Neuroscience 80: 545–557, 1997. [PubMed] [Google Scholar] 464. Richardson KA, Aston-Jones G. Lateral hypothalamic orexin/hypocretin neurons that project to ventral tegmental area are differentially activated with morphine preference. J Neurosci 32: 3809–3817, 2012. [PMC free article] [PubMed] [Google Scholar] 465. Richardson MJE, Gerstner W. Statistics of subthreshold neuronal voltage fluctuations due to conductance based synaptic shot noise. Chaos 16: 026106, 2006. [PubMed] [Google Scholar] 466. Richardson RT, DeLong MR. Context-dependent responses of primate nucleus basalis neurons in a go/no-go task. J Neurosci 10: 2528–2540, 1990. [PMC free article] [PubMed] [Google Scholar] 467. Riehle A, Requin J. Monkey primary motor and premotor cortex: single-cell activity related to prior information about direction and extent of an intended movement. J Neurophysiol 61: 534–549, 1989. [PubMed] [Google Scholar] 468. Ringach DL, Hawken MJ, Shapley R. Dynamics of orientation tuning in macaque primary visual cortex. Nature 387: 281–284, 1997. [PubMed] [Google Scholar] 469. Roberts S. Isolation of an internal clock. J Exp Psychol Anim Behav Proc 7: 242–268, 1981. [PubMed] [Google Scholar] 470. Robbins TW, Arnsten AFT. The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. Annu Rev Neurosci 32: 267–287, 2009. [PMC free article] [PubMed] [Google Scholar] 471. Robinson TE, Berridge KC. The neural basis for drug craving: an incentive-sensitization theory of addiction. Brain Res Rev 18: 247–291, 1993. [PubMed] [Google Scholar] 472. Robinson MJF, Berridge KC. Instant transformation of learned repulsion into motivational “wanting.” Curr Biol 23: 282–289, 2013. [PMC free article] [PubMed] [Google Scholar] 473. Rodriguez ML, Logue AW. Adjusting delay to reinforcement: comparing choice in pigeons and humans. J Exp Psychol Anim Behav Proc 14: 105–117, 1988. [PubMed] [Google Scholar] 474. Roelfsema PR, Tolboom M, Khayat PS. Different processing phases for features, figures, and selective attention in the primary visual cortex. Neuron 56: 785–792, 2007. [PubMed] [Google Scholar] 475. Roesch MR, Calu DJ, Esber GR, Schoenbaum G. Neural correlates of variations in event processing during learning in basolateral amygdala. J Neurosci 30: 2464–2471, 2010. [PMC free article] [PubMed] [Google Scholar] 476. Roesch MR, Olson CR. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J Neurophysiol 90: 1766–1789, 2003. [PubMed] [Google Scholar] 477. Roesch MR, Olson CR. Neuronal activity dependent on anticipated and elapsed delay in macaque prefrontal cortex, frontal and supplementary eye fields, and premotor cortex. J Neurophysiol 94: 1469–1497, 2005. [PubMed] [Google Scholar] 478. Roesch MR, Olson CR. Neuronal activity in orbitofrontal cortex reflects the value of time. J Neurophysiol 94: 2457–2471, 2005. [PubMed] [Google Scholar] 479. Roesch MR, Singh T, Brown PL, Mullins SE, Schoenbaum G. Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J Neurosci 29: 13365–13376, 2009. [PMC free article] [PubMed] [Google Scholar] 480. Rogers QR, Harper AE. Selection of a solution containing histidine by rats fed a histidine-imbalanced diet. J Comp Physiol Psychol 72: 66–71, 1970. [PubMed] [Google Scholar] 481. Roiser JP, de Martino B, Tan GCY, Kumaran D, Seymour B, Wood NW, Dolan RJ. A genetically mediated bias in decision making driven by failure of amygdala control. J Neurosci 29: 5985–5991, 2009. [PMC free article] [PubMed] [Google Scholar] 482. Roitman JD, Roitman MF. Risk-preference differentiates orbitofrontal cortex responses to freely chosen reward outcomes. Eur J Neurosci 31: 1492–1500, 2010. [PMC free article] [PubMed] [Google Scholar] 483. Roitman JD, Shadlen MN. Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. J Neurosci 22: 9475–9489, 2002. [PMC free article] [PubMed] [Google Scholar] 484. Roitman MF, Wheeler RA, Carelli RM. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45: 587–597, 2005. [PubMed] [Google Scholar] 485. Rolls ET, Critchley HD, Mason R, Wakeman EA. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J Neurophysiol 75: 1970–1981, 1996. [PubMed] [Google Scholar] 486. Rolls ET, Grabenhorst F, Parris BA. Warm pleasant feelings in the brain. NeuroImage 41: 1504–1513, 2008. [PubMed] [Google Scholar] 487. Rolls ET, Yaxley S, Sienkiewicz ZJ. Gustatory responses of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. J Neurophysiol 64: 1055–1066, 1990. [PubMed] [Google Scholar] 488. Romo R, Brody CD, Hernández A, Lemus L. Neuronal correlates of parametricworking memory in the prefrontal cortex. Nature 399: 470–473, 1999. [PubMed] [Google Scholar] 489. Romo R, Hernández A, Zainos A. Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron 41: 165–173, 2004. [PubMed] [Google Scholar] 490. Romo R, Hernández A, Zainos A, Lemus L, Brody CD. Neuronal correlates of decision-making in secondary somatosensory cortex. Nat Neurosci 5: 1217–1225, 2002. [PubMed] [Google Scholar] 491. Romo R, Schultz W. Neuronal activity preceding self-initiated or externally timed arm movements in area 6 of monkey cortex. Exp Brain Res 67: 656–662, 1987. [PubMed] [Google Scholar] 492. Romo R, Schultz W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. J Neurophysiol 63: 592–606, 1990. [PubMed] [Google Scholar] 493. Romo R, Schultz W. Role of primate basal ganglia and frontal cortex in the internal generation of movements. III. Neuronal activity in the supplementary motor area. Exp Brain Res 91: 396–407, 1992. [PubMed] [Google Scholar] 494. Rorie AE, Gao J, McClelland JL, Newsome WT. Integration of sensory and reward information during perceptual decision-making in lateral intraparietal cortex (LIP) of the macaque monkey. PLoS One 5: e9308, 2010. [PMC free article] [PubMed] [Google Scholar] 495. Rosenkranz JA, Grace AA. Dopamine-mediated modulation of odour-evoked amygdala potentials during pavlovian conditioning. Nature 417: 282–287, 2002. [PubMed] [Google Scholar] 496. Rossi MA, Fan D, Barter JW, Yin HH. Bidirectional modulation of substantia nigra activity by motivational state. PLoS One 8: e71598, 2013. [PMC free article] [PubMed] [Google Scholar] 497. Rothschild M, Stiglitz IA. Increasing risk: definition. J Econ Theory 2: 225–243, 1970. [Google Scholar] 498. Rutledge RB, Skandalia N, Dayan P, Dolan RJ. A computational and neural model of momentary subjective well-being. Proc Natl Acad Sci USA 111: 12252–12257, 2014. [PMC free article] [PubMed] [Google Scholar] 499. Salamone JD. The involvement of nucleus accumbens dopamine in appetitive and aversive motivation. Behav Brain Res 61: 117–133, 1994. [PubMed] [Google Scholar] 500. Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. Science 310: 1337–1340, 2005. [PubMed] [Google Scholar] 501. Samuelson P. A note on measurement of utility. Rev Econ Stud 4: 155–161, 1937. [Google Scholar] 502. Sato M, Hikosaka O. Role of primate substantia nigra pars reticulata in reward-oriented saccadic eye movement. J Neurosci 22: 2363–2373, 2002. [PMC free article] [PubMed] [Google Scholar] 503. Satoh T, Nakai S, Sato T, Kimura M. Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23: 9913–9923, 2003. [PMC free article] [PubMed] [Google Scholar] 504. Savage LJ. The Foundations of Statistics. New York: Wiley, 1954. [Google Scholar] 505. Sawaguchi T, Goldman-Rakic PS. D1 dopamine receptors in prefrontal cortex: involvement in working memory. Science 251: 947–950, 1991. [PubMed] [Google Scholar] 506. Schall JD. Neuronal activity related to visually guided saccadic eye movements in the supplementary motor area of rhesus monkeys. J Neurophysiol 66: 530–558, 1991. [PubMed] [Google Scholar] 507. Schall JD, Stuphorn V, Brown JW. Monitoring and control of action by the frontal lobes. Neuron 36: 309–322, 2002. [PubMed] [Google Scholar] 508. Schall JD, Thompson KG. Neural selection and control of visually guided eye movements. Annu Rev Neurosci 22: 241–259, 1999. [PubMed] [Google Scholar] 509. Scherberger H, andersen RA. Target selection signals for arm reaching in the posterior parietal cortex. J Neurosci 27: 2001–2012, 2007. [PMC free article] [PubMed] [Google Scholar] 510. Schmalfuss B. The random attractor of the stochastic Lorenz system. Z Angew Math Physik 48: 951–975, 1997. [Google Scholar] 511. Schmidt R, Leventhal DK, Mallet N, Chen F, Berke JD. Canceling actions involves a race between basal ganglia pathways. Nat Neurosci 16: 1118–1124, 2013. [PMC free article] [PubMed] [Google Scholar] 512. Schoenbaum G, Chiba AA, Gallagher M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci 19: 1876–1884, 1999. [PMC free article] [PubMed] [Google Scholar] 513. Schreiber CA, Kahneman D. Determinants of the remembered utility of aversive sounds. J Exp Psych 129: 27–42, 2000. [PubMed] [Google Scholar] 514. Schroeder T. Three Faces of Desire. Boston: MIT Press, 2004. [Google Scholar] 515. Schurger A, Jacobo Sitt JD D, Dehaene S. An accumulator model for spontaneous neural activity prior to self-initiated movement. J Proc Natl Acad Sci USA 109: E2904–E2913, 2012. [PMC free article] [PubMed] [Google Scholar] 516. Schultz W. Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. J Neurophysiol 56: 1439–1462, 1986. [PubMed] [Google Scholar] 517. Schultz W. Predictive reward signal of dopamine neurons. J Neurophysiol 80: 1–27, 1998. [PubMed] [Google Scholar] 518. Schultz W. Multiple dopamine functions at different time courses. Annu Rev Neurosci 30: 259–288, 2007. [PubMed] [Google Scholar] 519. Schultz W. Midbrain dopamine neurons: a retina of the reward system? In: Neuroeconomics: Decision Making and the Brain, edited by Glimcher PW, Camerer CF, Fehr E, Poldrack RA. New York: Academic, 2009, p. 323–329. [Google Scholar] 520. Schultz W. Potential vulnerabilities of neuronal reward, risk, and decision mechanisms to addictive drugs. Neuron 69: 603–617, 2011. [PubMed] [Google Scholar] 521. Schultz W, Apicella P, Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J Neurosci 13: 900–913, 1993. [PMC free article] [PubMed] [Google Scholar] 522. Schultz W, Apicella P, Romo R, Scarnati E. Context-dependent activity in primate striatum reflecting past and future behavioral events. In: Models of Information Processing in the Basal Ganglia, edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press, 1995, p. 11–28. [Google Scholar] 523. Schultz W, Apicella P, Scarnati E, Ljungberg T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J Neurosci 12: 4595–4610, 1992. [PMC free article] [PubMed] [Google Scholar] 524. Schultz W, Dayan P, Montague RR. A neural substrate of prediction and reward. Science 275: 1593–1599, 1997. [PubMed] [Google Scholar] 525. Schultz W, Romo R. Responses of nigrostriatal dopamine neurons to high intensity somatosensory stimulation in the anesthetized monkey. J Neurophysiol 57: 201–217, 1987. [PubMed] [Google Scholar] 526. Schultz W, Romo R. Neuronal activity in the monkey striatum during the initiation of movements. Exp Brain Res 71: 431–436, 1988. [PubMed] [Google Scholar] 527. Schultz W, Romo R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. J Neurophysiol 63: 607–624, 1990. [PubMed] [Google Scholar] 528. Schultz W, Romo R. Role of primate basal ganglia and frontal cortex in the internal generation of movements. I. Preparatory activity in the anterior striatum. Exp Brain Res 91: 363–384, 1992. [PubMed] [Google Scholar] 529. Searle JR. Intentionality. Cambridge, UK: Cambridge Univ. Press, 1983. [Google Scholar] 530. Seo H, Barraclough DJ, Lee D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb Cortex 17: i110-i117, 2007. [PubMed] [Google Scholar] 531. Seo H, Lee D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J Neurosci 27: 8366–8377, 2007. [PMC free article] [PubMed] [Google Scholar] 532. Seo H, Lee D. Behavioral and neural changes after gains and losses of conditioned reinforcers. J Neurosci 29: 3627–3641, 2009. [PMC free article] [PubMed] [Google Scholar] 533. Seo M, Lee E, Averbeck BB. Action selection and action value in frontal-striatal circuits. Neuron 74: 947–960, 2012. [PMC free article] [PubMed] [Google Scholar] 534. Seymour B, Daw ND, Roiser JP, Dayan P, Dolan R. Serotonin selectively modulates reward value in human decision-making. J Neurosci 32: 5833–5842, 2012. [PMC free article] [PubMed] [Google Scholar] 535. Shadlen MN, Newsome WT. Neural basis of a perceptual decision in the parietal cortex (Area LIP) of the rhesus monkey. J Neurophysiol 86: 1916–1936, 2001. [PubMed] [Google Scholar] 536. Sheafor PJ. Pseudoconditioned jaw movements of the rabbit reflect associations conditioned to contextual background cues. J Exp Psychol Anim Behav Proc 104: 245–260, 1975. [PubMed] [Google Scholar] 537. Sheafor PJ, Gormezano I. Conditioning the rabbit's (Oryctolagus cuniculus) jaw-movement response: US magnitude effects on URs, CRs, and pseudo-CRs. J Comp Physiol Psychol 81: 449–456, 1972. [PubMed] [Google Scholar] 538. Shefrin HM, Thaler RH. The behavioral life-cycle hypothesis. Econ Inq 26: 609–643, 1988. [Google Scholar] 539. Sheffield FD, Roby TB. Reward value of a non-nutritive sweet taste. J Comp Physiol Psychol 43: 471–481, 1950. [PubMed] [Google Scholar] 540. Shen W, Flajolet M, Greengard P, Surmeier DJ. Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321: 848–851, 2008. [PMC free article] [PubMed] [Google Scholar] 541. Shidara M, Aigner TG, Richmond BJ. Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18: 2613–2625, 1998. [PMC free article] [PubMed] [Google Scholar] 542. Shidara M, Richmond BJ. Anterior cingulate: single neuron signals related to degree of reward expectancy. Science 296: 1709–1711, 2002. [PubMed] [Google Scholar] 543. Shima K, Tanji J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282: 1335–1338, 1998. [PubMed] [Google Scholar] 544. Simmons JM, Richmond BJ. Dynamic changes in representations of preceding and upcoming reward in monkey orbitofrontal cortex. Cereb Cortex 18: 93–103, 2008. [PubMed] [Google Scholar] 545. Simon HA. Rational choice and the structure of the environment. Psychol Rev 63: 129–138, 1956. [PubMed] [Google Scholar] 546. Singh S, Lewis RL, Barto AG. Where do rewards come from? In: Proceedings of the 31st Annual Conference of the Cognitive Science Society, edited by Taatgen NA, van R, ijn H. Austin, TX: Cogn. Sci. Soc., 2009, p. 2601–2606. [Google Scholar] 547. Singh S, Lewis RL, Barto AG, Sorg J. Intrinsically motivated reinforcement learning: an evolutionary perspective. IEEE Trans Autonom Mental Dev 2: 70–82, 2010. [Google Scholar] 548. Smith AD, Bolam JP. The neural network of the basal ganglia as revealed by the study of synaptic connections of identified neurones. Trends Neurosci 13: 259–265, 1990. [PubMed] [Google Scholar] 549. Smith PL, Ratcliff R. Psychology and neurobiology of simple decisions. Trends Neurosci 27: 161–168, 2004. [PubMed] [Google Scholar] 550. So NY, Stuphorn V. Supplementary eye field encodes option and action value for saccades with variable reward. J Neurophysiol 104: 2634–2653, 2010. [PMC free article] [PubMed] [Google Scholar] 551. So NY, Stuphorn V. Supplementary eye field encodes reward prediction error. J Neurosci 32: 2950–2963, 2012. [PMC free article] [PubMed] [Google Scholar] 552. Solomon RL, Corbit JD. An opponent-process theory of motivation. Psychol Rev 81: 119–145, 1974. [PubMed] [Google Scholar] 553. Solomon PR, Vander Schaaf ER, Thompson RF, Weisz DJ. Hippocampus and trace conditioning of the rabbit's classically conditioned nictitating membrane response. Behav Neurosci 100: 729–744, 1986. [PubMed] [Google Scholar] 554. Soltani A, Lee D, Wang XJ. Neural mechanism for stochastic behaviour during a competitive game. Neur Networks 19: 1075–1090, 2006. [PMC free article] [PubMed] [Google Scholar] 555. Soon CS, Brass M, Heinze HJ, Haynes JD. Unconscious determinants of free decisions in the human brain. Nat Neurosci 11: 543–545, 2008. [PubMed] [Google Scholar] 556. Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G. Reward prediction error signaling in posterior dorsomedial striatum is action specific. J Neurosci 32: 10296–10305, 2012. [PMC free article] [PubMed] [Google Scholar] 557. Stanisor L, van der Togt C, Cyriel MA, Pennartz CMA, Roelfsema PR. A unified selection signal for attention and reward in primary visual cortex. Proc Natl Acad Sci USA 110: 9136–9141, 2013. [PMC free article] [PubMed] [Google Scholar] 558. Stanton SJ, Mullette-Gillman O'DA, McLaurin RE, Kuhn CM, LaBar KS, Platt ML, Huettel SA. Low- and high-testosterone individuals exhibit decreased aversion to economic risk. Psychol Sci 22: 447–453, 2011. [PMC free article] [PubMed] [Google Scholar] 559. Stauffer WR, Lak A, Bossaerts P, Schultz W. Economic choices reveal probability distortion in monkeys. J Neurosci 35: 3146–3154, 2015. [PMC free article] [PubMed] [Google Scholar] 560. Stauffer WR, Lak A, Schultz W. Dopamine reward prediction error responses reflect marginal utility. Curr Biol 24: 2491–2500, 2014. [PMC free article] [PubMed] [Google Scholar] 561. Stein RB. Some models of neuronal variability. Biophys J 7: 37–68, 1967. [PMC free article] [PubMed] [Google Scholar] 562. Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16: 966–973, 2013. [PMC free article] [PubMed] [Google Scholar] 563. Steinfels GF, Heym J, Strecker RE, Jacobs BL. Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res 258: 217–228, 1983. [PubMed] [Google Scholar] 564. Stevens CF. Inferences about membrane properties from electrical noise measurements. Biophys J 12: 1028–1047, 1972. [PMC free article] [PubMed] [Google Scholar] 565. Stone M. Models for choice reaction time. Psychometrika 25: 251–260, 1960. [Google Scholar] 566. Stopper CM, Tse MTL, David Montes DR R, Wiedman CR, Floresco SB. Overriding phasic dopamine signals redirects action selection during risk/reward decision making. Neuron 84: 177–189, 2014. [PubMed] [Google Scholar] 567. Strassberg AF, DeFelice LJ. Limitations of the Hodgkin-Huxley formalism: effects of single channel kinetics on transmembrane voltage dynamics. Neur Comp 5: 843–855, 1993. [Google Scholar] 568. Stuber GD, Klanker M, de Ridder B, Bowers MS, Joosten RN, Feenstra MG, Bonci A. Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons. Science 321: 1690–1692, 2008. [PMC free article] [PubMed] [Google Scholar] 569. Stuphorn V, Taylor TL, Schall JD. Performance monitoring by the supplementary eye field. Nature 408: 857–860, 2000. [PubMed] [Google Scholar] 570. Sugam JA, Day JJ, Wightman RM, Carelli RM. Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol Psychiat 71: 199–215, 2012. [PMC free article] [PubMed] [Google Scholar] 571. Sugrue LP, Corrado GS, Newsome WT. Matching behavior and the representation of value in the parietal cortex. Science 304: 1782–1787, 2004. [PubMed] [Google Scholar] 572. Sul JH, Kim H, Huh N, Lee D, Jung MW. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66: 449–460, 2010. [PMC free article] [PubMed] [Google Scholar] 573. Suri R, Schultz W. A neural network with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91: 871–890, 1999. [PubMed] [Google Scholar] 574. Sutton RS, Barto AG. Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88: 135–170, 1981. [PubMed] [Google Scholar] 575. Sutton RS, Barto AG. Reinforcement Learning. Cambridge, MA: MIT Press, 1998. [Google Scholar] 576. Tai LH, Lee AM, Benavidez N, Bonci A, Wilbrecht L. Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value. Nat Neurosci 15: 1281–1289, 2012. [PMC free article] [PubMed] [Google Scholar] 577. Takahashi T. A mathematical framework for probabilistic choice based on information theory and psychophysics. Med Hypoth 67: 183–186, 2006. [PubMed] [Google Scholar] 578. Takahashi YK, Roesch MR, Stalnaker TA, Haney RZ, Calu DJ, Taylor AR, Burke KA, Schoenbaum G. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62: 269–280, 2009. [PMC free article] [PubMed] [Google Scholar] 579. Takikawa Y, Kawagoe R, Hikosaka O. A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward maping. J Neurophysiol 92: 2520–2529, 2004. [PubMed] [Google Scholar] 580. Talwar SK, Xu S, Hawley ES, Weiss SA, Moxon KA, Chapin JK. Rat navigation guided by remote control. Nature 417: 37–38, 2002. [PubMed] [Google Scholar] 581. Tan CO, Bullock D. A local circuit model of learned striatal and dopamine cell responses under probabilistic schedules of reward. J Neurosci 28: 10062–10074, 2008. [PMC free article] [PubMed] [Google Scholar] 582. Tan KR, Yvon C, Turiault M, Mirzabekov JJ, Doehner J, Labouèbe G, Deisseroth K, Tye KM, Lüscher C. GABA neurons of the VTA drive conditioned place aversion. Neuron 73: 1173–1183, 2012. [PMC free article] [PubMed] [Google Scholar] 583. Tang KC, Low MJ, Grandy DK, Lovinger DM. Dopamine-dependent synaptic plasticity in striatum during in vivo development. Proc Natl Acad Sci USA 98: 1255–1260, 2001. [PMC free article] [PubMed] [Google Scholar] 584. Tanimoto H, Heisenberg M, Gerber B. Event timing turns punishment to reward. Nature 430: 983, 2004. [PubMed] [Google Scholar] 585. Teodorescu TR, Usher M. Disentangling decision models: from independence to competition. Psychol Rev 120: 1–38, 2013. [PubMed] [Google Scholar] 586. Tesauro G. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Comp 6: 215–219, 1994. [Google Scholar] 587. Thompson KG, Bichot NP, Schall JD. Dissociation of visual discrimination from saccade programming in macaque frontal eye field. J Neurophysiol 77:1046–1050, 1997. [PubMed] [Google Scholar] 588. Thompson KG, Hanes DP, Bichot NP, Schall JD. Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. J Neurophysiol 76: 4040–4055, 1996. [PubMed] [Google Scholar] 589. Thorndike EL. Animal Intelligence: Experimental Studies. New York: MacMillan, 1911. [Google Scholar] 590. Thut G, Schultz W, Roelcke U, Nienhusmeier M, Maguire RP, Leenders KL. Activation of the human brain by monetary reward. NeuroReport 8: 1225–1228, 1997. [PubMed] [Google Scholar] 591. Tindell AJ, Smith KS, Peciña S, Berridge KC, Aldridge JW. Ventral pallidum firing codes hedonic reward: when a bad taste turns good. J Neurophysiol 96: 2399–2409, 2006. [PubMed] [Google Scholar] 592. Thorpe SJ, Rolls ET, Maddison S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp Brain Res 49: 93–115, 1983. [PubMed] [Google Scholar] 593. Tinklepaugh OL. An experimental study of representation factors in monkeys. J Comp Psychol 8: 197–236, 1928. [Google Scholar] 594. Toan DL, Schultz W. Responses of rat pallidum cells to cortex stimulation and effects of altered dopaminergic activity. Neuroscience 15: 683–694, 1985. [PubMed] [Google Scholar] 595. Tobler PN, Christopoulos GI, O'Doherty JP, Dolan RJ, Schultz W. Neuronal distortions of reward probability without choice. J Neurosci 28:11703–11711, 2008. [PMC free article] [PubMed] [Google Scholar] 596. Tobler PN, Christopoulos GI, O'Doherty JP, Dolan RJ, Schultz W. Risk-dependent reward value signal in human prefrontal cortex. Proc Natl Acad Sci USA 106: 7185–7190, 2009. [PMC free article] [PubMed] [Google Scholar] 597. Tobler PN, Dickinson A, Schultz W. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23: 10402–10410, 2003. [PMC free article] [PubMed] [Google Scholar] 598. Tobler PN, Fiorillo CD, Schultz W. Adaptive coding of reward value by dopamine neurons. Science 307: 1642–1645, 2005. [PubMed] [Google Scholar] 599. Tobler PN, O'Doherty JP, Dolan R, Schultz W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J Neurophysiol 97: 1621–1632, 2007. [PMC free article] [PubMed] [Google Scholar] 600. Tremblay L, Hollerman JR, Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J Neurophysiol 80: 964–977, 1998. [PubMed] [Google Scholar] 601. Tremblay L, Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 398: 704–708, 1999. [PubMed] [Google Scholar] 602. Tremblay L, Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol 83: 1864–1876, 2000. [PubMed] [Google Scholar] 603. Tremblay L, Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J Neurophysiol 83: 1877–1885, 2000. [PubMed] [Google Scholar] 604. Tsai CT, Nakamura S, Iwama K. Inhibition of neuronal activity of the substantia nigra by noxious stimuli and its modification by the caudate nucleus. Brain Res 195: 299–311, 1980. [PubMed] [Google Scholar] 605. Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 324: 1080–1084, 2009. [PMC free article] [PubMed] [Google Scholar] 606. Tsujimoto S, Genovesio A, Wise SP. Monkey orbitofrontal cortex encodes response choices near feedback time. J Neurosci 29: 2569–2574, 2009. [PMC free article] [PubMed] [Google Scholar] 607. Umeno MM, Goldberg ME. Spatial processing in the monkey frontal eye field. I. Predictive visual responses. J Neurophysiol 78: 1373–1383, 1997. [PubMed] [Google Scholar] 608. Ungless MA, Magill PJ, Bolam JP. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303: 2040–2042, 2004. [PubMed] [Google Scholar] 609. Usher M, McClelland JL. The time course of perceptual choice: the leaky, competing accumulator model. Psych Rev 108: 550–592, 2011. [PubMed] [Google Scholar] 610. Valenza E, Simion F, Macchi Cassia V, Umiltà C. Face Preference at Birth. J Exp Psych Hum Percept Perform 22: 892–903, 1996. [PubMed] [Google Scholar] 611. Van Vreeswijk C, Sompolinsky H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726, 1996. [PubMed] [Google Scholar] 612. Van Wolkenten M, Brosnan SF, de Waal F. Inequity responses of monkeys modified by effort. Proc Nat Acad Sci USA 104: 18854–18859, 2007. [PMC free article] [PubMed] [Google Scholar] 613. Van Zessen R, Phillips JL, Budygin EA, Stuber GD. Activation of VTA GABA neurons disrupts reward consumption. Neuron 73: 1184–1194, 2012. [PMC free article] [PubMed] [Google Scholar] 614. Vickers D. Evidence for an accumulator model of psychophysical discrimination. Ergonomics 13: 37–58, 1970. [PubMed] [Google Scholar] 615. Vickery TJ, Chun MM, Lee D. Ubiquity and specificity of reinforcement signals throughout the human brain. Neuron 72: 166–177, 2011. [PubMed] [Google Scholar] 616. Vijayraghavan S, Wang M, Birnbaum SG, Williams GV, Arnsten AFT. Inverted-U dopamine D1 receptor actions on prefrontal neurons engaged in working memory. Nat Neurosci 10: 376–384, 2007. [PubMed] [Google Scholar] 617. Von Neumann J, Morgenstern O. The Theory of Games and Economic Behavior. Princeton, NJ: Princeton Univ. Press, 1944. [Google Scholar] 618. Waelti P, Dickinson A, Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412: 43–48, 2001. [PubMed] [Google Scholar] 619. Wald A, Wolfowitz J. Optimum character of the sequential probability ratio test. Ann Math Statist 19: 326–339, 1947. [Google Scholar] 620. Wallis JD, Miller EK. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci 18: 2069–2081, 2003. [PubMed] [Google Scholar] 621. Wanat MJ, Kuhnen CM, Phillips PEM. Delays conferred by escalating costs modulate dopamine release to rewards but not their predictors. J Neurosci 30: 12020–12027, 2010. [PMC free article] [PubMed] [Google Scholar] 622. Wang XJ. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36: 955–968, 2002. [PubMed] [Google Scholar] 623. Wang XJ. Decision making in recurrent neuronal circuits. Neuron 60: 215–234, 2008. [PMC free article] [PubMed] [Google Scholar] 624. Wang LP, Li F, Wang D, Xie K, Wang D, Shen X, Tsien JZ. NMDA receptors in dopaminergic neurons are crucial for habit learning. Neuron 72: 1055–1066, 2011. [PMC free article] [PubMed] [Google Scholar] 625. Watabe-Uchida M, Zhu L, Ogawa SK, Vamanrao A, Uchida N. Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 74, 858–873, 2012. [PubMed] [Google Scholar] 626. Watanabe M. Prefrontal unit activity during associative learning in the monkey. Exp Brain Res 80: 296–309, 1990. [PubMed] [Google Scholar] 627. Watanabe M. Reward expectancy in primate prefrontal neurons. Nature 382: 629–632, 1996. [PubMed] [Google Scholar] 628. Watanabe M, Hikosaka K, Sakagami M, Shirakawa SI. Coding and monitoring of behavioral context in the primate prefrontal cortex. J Neurosci 22: 2391–2400, 2002. [PMC free article] [PubMed] [Google Scholar] 629. Watson KK, Platt ML. Social signals in primate orbitofrontal cortex. Curr Biol 22: 2268–2273, 2012. [PMC free article] [PubMed] [Google Scholar] 630. Weber BJ, Chapman GB. Playing for peanuts: why is risk seeking more common for low-stakes gambles? Organiz Behav Human Dec Proc 97: 31–46, 2005. [Google Scholar] 631. Weber EU, Johnson EJ. Decisions under uncertainty: psychological, economic, and neuroeconomic explanations of risk preference. In: Neuroeconomics, edited by Glimcher PW, Johnson EJ, Camerer CF, Fehr E, Poldrack RA. London: Academic, 2009. [Google Scholar] 632. Weber EU, Milliman RA. Perceived risk attitudes: relating risk perception to risky choice. Management Sci 43: 123–144, 1997. [Google Scholar] 633. Weber EU, Shafir S, Blais AR. Predicting risk sensitivity in humans and lower animals: risk as variance or coefficient of variation. Psychol Rev 111: 430–445, 2004. [PubMed] [Google Scholar] 634. Wegner DM. The Illusion of Conscious Will. Cambridge, MA: MIT Press, 2002. [Google Scholar] 635. West EA, Forcelli PA, McCuea DL, Malkova L. Differential effects of serotonin-specific and excitotoxic lesions of OFC on conditioned reinforcer devaluation and extinction in rats. Behav Brain Res 246: 10–14, 2013. [PMC free article] [PubMed] [Google Scholar] 636. Weinrich M, Wise SP. The premotor cortex of the monkey. J Neurosci 2: 1329–1345, 1982. [PMC free article] [PubMed] [Google Scholar] 637. Wickens J, Kötter R. Cellular models of reinforcement. In: Models of Information Processing in the Basal Ganglia, edited by Houk JC, Davis JL, Beiser DG. Cambridge, MA: MIT Press, 1995, p. 187–214. [Google Scholar] 638. Wightman RM, Robinson DL. Transient changes in mesolimbic dopamine and their association with “reward.” J Neurochem 82: 721–735, 2002. [PubMed] [Google Scholar] 639. Williams ZM, Eskandar EN. Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat Neurosci 4: 562–568, 2006. [PubMed] [Google Scholar] 640. Wise RA. Brain reward circuitry: insights from unsensed incentives. Neuron 36: 229–240, 2002. [PubMed] [Google Scholar] 641. Witten IB, Steinberg EE, Lee SY, Davidson TJ, Zalocusky KA, Brodsky M, Yizhar O, Cho SL, Gong S, Ramakrishnan C, Stuber GD, Tye KM, Janak PH, Deisseroth K. Recombinase-driver rat lines: tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron 72: 721–733, 2011. [PMC free article] [PubMed] [Google Scholar] 642. Yagishita S, Hayashi-Takagi A, Ellis-Davies GCR, Urakubo H, Ishii S, Kasai H. A critical time window for dopamine actions on the structural plasticity of dendritic spines. Science 345: 1616–1620, 2014. [PMC free article] [PubMed] [Google Scholar] 643. Yang T, Shadlen MN. Probabilistic reasoning by neurons. Nature 447: 1075–1080, 2007. [PubMed] [Google Scholar] 644. Yamada H, Tymula A, Louie K, Glimcher PW. Thirst-dependent risk preferences in monkeys identify a primitive form of wealth. Proc Natl Acad Sci USA 110: 15788–15793, 2013. [PMC free article] [PubMed] [Google Scholar] 645. Yasuda M, Yamamoto S, Hikosaka O. Robust representation of stable object values in the oculomotor basal ganglia. J Neurosci 32: 16917–16932, 2012. [PMC free article] [PubMed] [Google Scholar] 646. Yin HH, Ostlund SB, Knowlton BJ, Balleine BB. The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 23: 513–523, 2005. [PubMed] [Google Scholar] 647. Yoshida K, Saito N, Iriki A, Isoda M. Social error monitoring in macaque frontal cortex. Nat Neurosci 15, 1307–1312, 2012. [PubMed] [Google Scholar] 648. Zhang JC, Lau PM, Bi GQ. Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses. Proc Natl Acad Sci USA 106: 1328–1333, 2009. [PMC free article] [PubMed] [Google Scholar] 649. Zweifel LS, Argilli E, Bonci A, Palmiter R. Role of NMDA receptors in dopamine neurons for plasticity and addictive behaviors. Neuron 59: 486–496, 2008. [PMC free article] [PubMed] [Google Scholar] 650. Zweifel LS, Parker JG, Lobb CJ, Rainwater A, Wall VZ, Fadok JP, Darvas M, Kim MJ, Mizumori SJ, Paladini CA, Philipps PEM, Palmiter R. Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior. Proc Natl Acad Sci USA 106, 7281–7288, 2009. [PMC free article] [PubMed] [Google Scholar] |
Humans do not have innate behaviors. please select the best answer from the choices provided t f
Urheberrechte © © 2024 WIKIFI Inc.
