Email Icon Facebook icon Twitter Icon GooglePlus Icon Contact

User Top Menu

Working Memory Deficit Hampers Reinforcement Learning in Schizophrenia

13 Oct 2014

October 14, 2014. The cognitive impairments in schizophrenia may be more tangled than previously appreciated, according to a study published October 8 in the Journal of Neuroscience. Jim Gold of the University of Maryland, Baltimore, and Michael Frank of Brown University in Providence, Rhode Island, teamed up to parse the different components involved in learning stimulus-response associations—a test of reinforcement learning. Using a novel behavioral task combined with computational modeling, the researchers found that the trouble people with schizophrenia had in learning the associations stemmed from deficits in working memory rather than in reinforcement learning itself.

The findings indicate overlaps between processes that might pertain to motivation—and the loss of it—in schizophrenia.

"We've tended to say that motivation is about reinforcement, separate from the cognitive functions," said Deanna Barch of Washington University, St. Louis, Missouri, who was not involved in the study. "But this study shows we really need to think about motivation and cognition in a more integrated fashion."

Previous work by Barch and others has shown that people with schizophrenia struggle to do reinforcement learning tasks. The main system governing reinforcement learning resides in the basal ganglia, where it makes predictions about outcomes associated with particular actions and takes feedback about what actually happened to modify behavior. The role of dopamine is to signal prediction errors. Problems with this process could result in faulty reward processing and, perhaps, the lack of motivation characterizing negative symptoms.

But people with schizophrenia also show impaired working memory, which helps keep multiple pieces of information in mind for imminent use and which is headquartered in the prefrontal cortex. Problems with working memory could also make it difficult to do reinforcement learning tasks. For example, remembering a phone number could involve mentally rehearsing the sequence of numbers, a quick but capacity-limited process invoking working memory, and typing the number out on a keypad multiple times, a slower, incremental process that calls reinforcement learning into play.

Hints of a prefrontal contribution to reinforcement learning emerged in an earlier study from Gold and Frank that also involved computational modeling (see SRF related news report and interview with Gold). There, the researchers found that people with schizophrenia had problems representing value—something the prefrontal cortex would do.

By finding a role for working memory, the new study comes up with more evidence for the prefrontal cortex’s involvement. But the researchers do not go so far as to say that all problems with reinforcement learning in schizophrenia come from impaired working memory.

"It’s possible the task we used put a premium on the role of working memory," Gold told SRF. "But the experiment provides a proof of principle that working memory problems likely do contribute to reinforcement learning problems."

Set size

The study used a task developed by first author Anne Collins and Michael Frank to disentangle working memory from reinforcement learning (Collins and Frank, 2012). This task asks people to figure out by trial and error the correct response (button press) to a visual stimulus—a standard reinforcement learning paradigm. But their task added a twist to recruit working memory: it had people work out the stimulus-response pairings in ever increasing set sizes. For example, study participants had to work out responses to a set of two stimuli in parallel, then three, then four, all the way up to six.

Gold tested 49 people with schizophrenia and 36 healthy controls on this task. People with schizophrenia could learn the pairings, though they were slower than controls, and for larger set sizes did not achieve the same level of accuracy as controls.

Taking the behavioral data on learning rates, decay (or forgetting), working memory capacity, and other variables for each person, Collins and Frank then set about exploring, computationally, the mix of working memory and reinforcement learning processes that could account for the data. The best fit required both processes in schizophrenia and in controls; however, people with schizophrenia relied less on working memory, yet showed normal reinforcement learning engagement compared to controls.

"Without the computational part of this project, they wouldn't have been able to tease apart these mechanisms," Barch said.

The clear role for working memory might have come at the expense of a role for reinforcement learning, Gold said. Participants received "deterministic" feedback, meaning they were told whether their button presses were right or wrong on each trial. This helped provide clear answers about working memory content, but may not have engaged reinforcement learning processes as much as a "probabilistic" feedback setup, which forces the system to integrate outcomes over time.

Among participants with schizophrenia, the researchers did not find a correlation between working memory capacity and negative symptoms—a bit of a conundrum for the idea that problems with reinforcement learning underlie apathy. The subjects did not show as wide a variation in working memory capacity as might be needed to see such a relationship, Gold said. All participants were medicated as well, which further muddied the relationship between dopamine signaling and reinforcement learning.

Gold is looking toward imaging in the near future. "We have a solid behavioral and modeling effect, so it would be nice to nail down which brain regions are involved," he said.—Michele Solis.

Reference:

Collins AG, Brown JK, Gold JM, Waltz JA, Frank MJ. Working memory contributions to reinforcement learning impairments in schizophrenia. J Neurosci. 2014 Oct 8;34(41):13747-56. Abstract