Dopamine: More than just reward

June 14, 2022

Written by: Catrina Hacker

If you’ve heard about dopamine, you probably also know it plays an important role in signaling reward. Lots of popular media has been dedicated to tips for how to increase dopamine levels and people sometimes joke about getting “a hit of dopamine” to rewarding experiences. But dopamine is a complex neurotransmitter with many different functions, including shaping movement and memory¹ . Neuroscientists have noticed for a long time that rewards tend to be associated with an increase in dopamine, but dopamine can also be released during stressful events. So, contrary to popular belief, dopamine is playing a more complicated role than just signaling the pleasure of receiving a reward.

While neuroscientists are still working toward understanding the complexities of the role that dopamine plays in the brain, there is general agreement about at least one thing that dopamine might be signaling: expectation of reward. To illustrate this, think about receiving a paycheck. You probably expect to see a certain amount of money and any big deviations up or down from that number would be surprising. Alternatively, hearing the song your favorite ice cream truck plays might make you suddenly crave ice cream as you expect to be getting a treat.

There are many ways we can learn to expect reward, but neuroscientists have been able to learn a lot about the expectation of reward using a relatively simple training paradigm called classical conditioning that makes it easy to train lab animals. Classical conditioning was first implemented by Ivan Pavlov in 1897. While working with dogs, Pavlov noticed that just the sight of their handler was enough to make them salivate in anticipation of being fed, just like hearing the song from your favorite ice cream truck might make you crave ice cream. Pavlov realized that he could teach the dogs to salivate in response to any neutral stimulus as long as he consistently paired it with reward². To demonstrate this, Pavlov trained the dogs to salivate when they heard a bell (Figure 1). Before training, the dogs would salivate when they saw their food (salivation is a natural response in preparation to eat), but not when they heard a bell ring. Here, we refer to the food as the unconditioned stimulus, because the dogs respond this way without any training. During training, Pavlov consistently rang a bell immediately before presenting the food. Whereas before training the dogs did not respond to the ringing bell, after training the dogs would salivate any time the bell was rung, even when no food was presented. We refer to the bell as the conditioned stimulus, because after training (also called conditioning) the dogs respond by salivating. In the lab, different types of animals can be trained to learn the same kinds of associations by being trained to respond to a conditioned stimulus paired with rewards like juice and sugar. So how does the brain represent this association? All signs point to dopamine.

To better understand what role dopamine might be playing in signaling reward and expectation, a group of neuroscientists recorded the activity of dopaminergic neurons (neurons that release dopamine) while training an animal to associate the sound of a beep with a juice reward (Figure 2)³. Before training, dopaminergic neurons would fire strongly when juice, the unconditioned stimulus, was dispensed at random and unexpected times. Importantly, the neurons always increased their activity immediately after the animal received the juice, and never before or long after (Figure 2). In the figure, the relevant times we anticipate a response from the neuron are highlighted with yellow boxes and the more times the neuron spikes (shown by the black lines) the stronger it is responding.

During training, the scientists repeatedly played a beep and then dispensed the juice after a consistent amount of time. After training, they noticed that the dopaminergic neurons were responding differently than before training. The neurons now increased their activity in response to the beep and not when juice was dispensed (Figure 2). This supports the idea that dopaminergic neurons are activated in response to unexpected reward. Whereas getting juice was unexpected before training, getting juice after a beep became expected during training. Even though the juice itself is still rewarding, the dopamine is released before receiving the reward because the juice is now expected. The neurons now respond to the unexpected beep in anticipation of the rewarding juice they expect to get.

The neuroscientists made one other observation about the response of the dopaminergic neurons after training that further supports the idea that the activity of these neurons represents the expectedness of a reward. To understand this observation, it’s important to note that neurons are rarely silent. They are always firing at a certain low level that we refer to as baseline activity (Figure 2). So, when I say that the neurons respond to the juice or the beep, what I mean is that they are firing more than the baseline. What the neuroscientists also observed was that after training when they played a beep and didn’t dispense juice, the neuron still increased its activity when the beep was played, but it also decreased its activity just after the time the juice would normally have been dispensed (Figure 2). In other words, the dopaminergic neurons increased their activity to signal unexpected reward, but they also decreased their activity to signal that an expected reward was missing. Because of this, this phenomenon is known as reward prediction error because the dopaminergic neuronsare signaling when there is an error in the brain’s prediction of a reward. Another way to think about this is by returning to the example of looking at your paycheck. Your dopaminergic neurons would fire more if you were paid more than you expected, and less if you were paid less than you expected.

All of this points to an important role for dopamine in changing how the brain responds to things depending on whether they are more or less expected. While this might seem like a trivial observation, representing the expectedness of a stimulus is a very clever way for the brain to be more efficient. While the brain makes up 2% of our body mass, it uses 20% of the body’s energy⁴.  This is pretty remarkable when we consider the complexity of what our brain accomplishes every second of the day. Reward prediction error is one of many signals the brain uses to maximize efficiency and make this possible. By signaling when rewards (or lack of them) deviate from our expectation, dopaminergic neurons help the brain prioritize important things that might be worth investigating further.

References

1.         Iversen, S. D. & Iversen, L. L. Dopamine: 50 years in perspective. Trends Neurosci. 30, 188–193 (2007).

2.         Classical Conditioning Involves Associating Two Stimuli. in Principles of Neural Science (eds. Kandel, E. R., Schwartz, J. H. & Jessell, T. M.) 1240–1242 (McGraw-Hill, 2000).

3.         Schultz, W., Dayan, P. & Montague, P. R. A Neural Substrate of Prediction and Reward. Science 275, 1594–1599 (1997).

4.         Mink, J. W., Blumenschine, R. J. & Adams, D. B. Ratio of central nervous system to body metabolism in vertebrates: its constancy and functional basis. Am. J. Physiol.-Regul. Integr. Comp. Physiol. 241, R203–R212 (1981).

Cover photo from Michael G on Unsplash.