Schizophrenia Research Forum - A Catalyst for Creative Thinking

Studies Spotlight Basic Sound Processing Problems in Schizophrenia

7 May 2007. A fortunate conflux of studies reveals functional and structural deficits in the early processing of information about sounds in schizophrenia. Their findings may shed light both on why people with the illness experience auditory hallucinations and have difficulty reading other people’s emotions and meaning in conversation. The studies also counter the view that these symptoms stem primarily from problems with higher-level functions such as attention and memory.

In one of the studies, Robert A. Sweet of the University of Pittsburgh, Pennsylvania, and associates find that the brains of people with schizophrenia have relatively sparse axon terminals in areas responsible for processing basic auditory information. Daniel C. Javitt, at the Nathan S. Kline Institute for Psychiatric Research in Orangeburg, New York, and colleagues report that white matter anomalies in early auditory processing regions in schizophrenia may underlie the lack of ability to discern emotion, and perhaps other kinds of meaning, in speech. Two other studies, directed by Judith M. Ford of Yale University, New Haven, Connecticut, show how the brain’s mechanism for differentiating self- versus other-generated speech malfunctions in schizophrenia.

Unsound Structures, Pitch Troubles, Tied to Social Deficits
People with schizophrenia do poorly at identifying speakers’ emotions from their tone of voice. In the March American Journal of Psychiatry, Javitt, first author David I. Leitman, and colleagues, present two studies looking at impairments in the ability to detect prosody, or language-related rhythm and tones, in schizophrenia.

The first focused on affective prosody and brain structure in 19 subjects with schizophrenia or schizoaffective disorder and 19 healthy controls. Leitman and colleagues used fractional anisotropy derived from diffusion tensor magnetic resonance imaging to gauge the diffusion of water along axonal fibers and myelin, a reflection of their structural integrity.

Subjects with schizophrenia did worse than controls on tasks that involved spotting wrong notes in common melodies and decoding emotions from the speaker’s tone of voice. Javitt says people with schizophrenia “can’t detect emotions from people’s voices just because they can’t detect the music in the voices.”

On both tasks, poor performance was associated with lower anisotropy in early auditory sensory pathways. Not surprisingly, it also correlated with lower anisotropy in pathways to the amygdala, an area thought to play a large role in emotion, and for the emotion identification task, in the orbitofrontal cortex and corpus callosum. The findings “suggest that functional and structural deficits within early sensory regions contribute to the overall pattern of cognitive dysfunction in schizophrenia,” Leitman and colleagues write.

The second study tested whether deficits in musical pitch perception would coincide with impairments in detecting nonaffective prosody. To assess the latter, Leitman and colleagues asked 24 subjects with schizophrenia or schizoaffective disorder and 17 healthy controls to discriminate between sentences that differed in emphasis between subject and object, or between statements and questions based on tone of voice.

The performance of subjects with schizophrenia lagged behind that of controls on all prosody tasks. Relationships between pitch perception and different kinds of prosodic detection imply that tone-processing deficits may hinder recognition of emotional and nonemotional meaning in schizophrenia. Javitt says it was once thought that individuals with the disease are in “a kind of confusional state,” but “it’s only recently that people are coming around to saying maybe the information just isn’t getting in properly.”

Structures Lacking in Auditory Cortex
In the second paper, published in the April 1 issue of Biological Psychiatry, Sweet and colleagues at the University of Pittsburgh also provide evidence of structural abnormalities in parts of the auditory cortex involved in low-level processing (see also a commentary on this article, Shi, 2007). Prior research by his group found that people with schizophrenia have pyramidal cell bodies of abnormally low volume in deep layer 3 of Brodmann's areas 41 and 42, but not layer 5 of area 42 (Sweet et al., 2003; 2004). The layer 3 cells provide feedforward connections that help rush sensory input from one cortical region to another; layer 5 cells provide feedback circuits that allow contextual processing.

Since excitatory input from feedforward projections supports cell size and growth; “this pattern of abnormalities suggests that structural alterations in auditory cortex of subjects with schizophrenia are present in feedforward, but not feedback, circuits,” Sweet and colleagues write. They tested this hypothesis using brain tissue from autopsies of 15 people with schizophrenia or schizoaffective disorder and 15 comparison subjects matched on age, gender, and tissue storage duration.

Compared to controls, subjects with schizophrenia had 13.6 percent lower density of a marker of axon terminals in deep layer 3 of area 41, which gets input from the thalamus and from within itself. In contrast, the marker indicated similar terminal densities between patients and controls in layer 1 of area 41, an endpoint for feedback projections that start and stop in the cortex, but also in deep layer 3 of area 42, a higher level of the feedforward pathway.

In order to address the possible confound of antipsychotic drug treatment, the researchers replicated the experiments in nonhuman primates. They found no evidence that haloperidol exposure could affect the density of axon terminals within auditory cortical areas.

Area 41, the primary auditory cortex, helps distinguish pure tones, localize sounds, and define beginnings and ends of sounds; area 42 responds to more complex groupings of sounds. Sweet says the results point to “reductions in the structures that support the rapid spread of auditory sensory information in the earliest stages of processing in the cerebral cortex.”

Flawed Response to "Own" Speech
Auditory hallucinations may result when the brain cannot tell its own thoughts from others’ voices. In the March Archives of General Psychiatry, Ford, first author Theda H. Heinks-Maldonado, of Albert-Ludwigs-Universitaet in Germany, Daniel H. Mathalon, also of Yale, and colleagues relate hallucinations to a failure of the brain to respond differently to self- versus other-generated actions.

Before someone talks, a neural signal travels from speech production areas to auditory cortex. It carries a prediction of the speech sounds based on a copy of the motor command and triggers a “corollary discharge.” If the corollary discharge matches what the speaker hears, as it should when the sound is self-produced, the sensory experience is reduced. The sensory experience thus carries a tag declaring it "self-produced," which presumably helps us avoid being distracted by our own talking.

Previous work by this group showed that the N100 component of the electroencephalogram, which Ford says “may represent one of the first representations of the auditory stimulus in the cortex,” typically responds with less amplitude to speaking than listening (Ford et al., 2005). In schizophrenia, this suppression may not occur, perhaps due to a faulty corollary discharge system. To investigate further, Heinks-Maldonado and colleagues compared N100 responses to speaking and listening in 20 men with schizophrenia, half of whom regularly heard voices, and 17 age-matched healthy men.

In the speaking task, the men said “ah” every few seconds and received instantaneous feedback over headphones. Feedback varied between their own voice and another speaker's (or "alien") voice, both either unchanged or with the pitch shifted down. The subjects’ own sounds were recorded and played back during the listening task. The subjects then indicated whose voice they heard.

Hallucinators made more voice misattributions in all conditions during talking, and unexpectedly, while listening. During talking, controls showed a dampened N100 response, especially over the left hemisphere, to their own unchanged voice compared to the other voice feedback conditions. Some non-hallucinating patients also showed normal suppression, but hallucinators’ N100 amplitude did not differentiate the four types of feedback, suggesting that their central nervous system failed to temper its response to self-generated sounds. Less suppression was linked to more errors during speaking (r = 0.32, P = .04, 1-tailed) and, in the left hemisphere, to worse hallucinations (r = 0.49, P = .02).

Ford explained, “In healthy, normal controls, the corollary discharge is a precise representation of what you expect, and there is suppression when what you hear is what you expected. In schizophrenics, that relationship is off; and it is no longer precise.”

Neurons Out of Sync
Finally, in the March American Journal of Psychiatry, Ford, Mathalon, and colleagues reasoned that if the motor and sensory systems together generate the corollary discharge, then the involved neurons should become entrained to oscillate in sync, or simultaneously fire at the same frequency, before speaking. They expected less synchronous firing in schizophrenia. To test this theory, they gathered EEG data from 24 subjects with schizophrenia or schizoaffective disorder and 25 healthy controls as they spoke and listened. They measured neural synchrony across trials.

Controls showed more synchrony before talking than during listening. This difference was found in subjects with schizophrenia as well, but to a lesser extent, corroborating the notion of an inexact corollary discharge in schizophrenia. Those with the worst hallucinations showed the least pre-speech synchrony. Preparatory synchrony and suppressed N100 response were related in control subjects but not in those with schizophrenia.

“We suggest that this premovement burst of synchronous neural activity is a reflection of the forward model preparing the CNS for the sensory consequences of its own actions,” Ford and colleagues write. They note that “a wide range” of schizophrenia symptoms “could result from a deficient message being sent to the sensory cortex that the action is ‘self-generated.’”

Together, these four papers attest to fundamental problems in the processing of information about sounds that may contribute to symptoms previously ascribed to association areas, frontal cortex, or limbic areas. As far as the researchers could tell, antipsychotic medication did not cause the deficits. However, most of the studies included few female subjects, leaving their generalizability unclear.

In conclusion, Mathalon says “schizophrenics may have difficulty with auditory discrimination in general,” from appreciating prosody to listening to voices. To him, “the results suggest that during speech production there is an additional, and perhaps, independent, deficiency.”—Victoria L. Wilcox.

Ford JM, Roach BJ, Faustman WO, Mathalon DH. Synch before you speak: Auditory hallucinations in schizophrenia. Am J Psychiatry. 2007;164:458-466. Abstract

Heinks-Maldonado TH, Mathalon DH, Houde JF, Gray M, Faustman WO, Ford JM. Relationship of imprecise corollary discharge in schizophrenia to auditory hallucinations. Arch Gen Psychiatry. 2007:64:286-296. Abstract

Leitman DI, Hoptman MJ, Foxe JJ, Saccente E, Wylie GR, Nierenberg J, Jalbrzikowski M, Lim KO, Javitt DC. The neural substrates of impaired prosodic detection in schizophrenia and its sensorial antecedents. Am J Psychiatry. 2007;164:474-482. Abstract

Sweet RA, Bergen SE, Sun Z, Marcsisin MJ, Sampson AR, Lewis DA. Anatomical evidence of impaired feedforward auditory processing in schizophrenia. Biol Psychiatry. 2007:61:854-864. Abstract

Comments on Related News

Related News: In Sync—Orchestrating Perfect Harmony in Neuronal Networks

Comment by:  Kevin Spencer (Disclosure)
Submitted 9 February 2006
Posted 9 February 2006
  I recommend the Primary Papers

Related News: Poor Working Memory Starts Early in Processing Pipeline

Comment by:  Steven Silverstein
Submitted 19 June 2011
Posted 19 June 2011
  I recommend the Primary Papers

This new paper by Dias et al. is important in at least two respects. First, it is further evidence of visual processing impairment in schizophrenia. Second, and perhaps more importantly, as noted by the authors, the findings provide more evidence that schizophrenia is not primarily a disorder of cognitive control or prefrontal cortex function. That is, while schizophrenia does involve those impairments, it appears likely that they can be viewed as 1) being affected by inadequate representations emerging from more basic aspects of processing such as perception; and 2) being due to a primary disorder of neural network formation and maintenance—a widespread failure that affects all aspects of cognitive functioning. These points are discussed in detail in a forthcoming theme section of Schizophrenia Bulletin on Vision Science and Schizophrenia Research.

As noted by Javitt in the article summary, the findings by Dias et al. suggest “that not only are the frontal activation deficits in schizophrenia due to sensory deficits, but, in fact, the behavioral deficits are really reflecting the sensory dysfunction more than the higher frontal dysfunction.” While this statement may seem controversial, or even heretical, it is, nevertheless, also supported by data.

For example, the study is consistent with past behavioral, ERP, and fMRI data showing how working memory deficits in schizophrenia are downstream effects of inadequate sensory and perceptual processing (e.g., Haenschel et al., 2007; Silverstein et al., 2005). In fact, only in cases where stimuli were used for which it is known that schizophrenia patients do not have a problem (e.g., basic color perception) has it been found that perceptual factors do not limit working memory function (e.g., Gold et al., 2010).

On the broader point noted above, regarding the benefits of a greater understanding of visual processing impairments in schizophrenia, there are several relevant considerations. One is that many, if not all, of the processes typically studied in schizophrenia can be more easily and clearly studied using paradigms from vision science. For example, context processing (the focus of the study by Dias et al.) occurs in visual cortex as well (at several levels, including gain control, center-surround modulation, motion perception, collinear facilitation, and other effects of spatial context on perception, and effects of task context such as the order in which stimuli or conditions are presented). Moreover, other functions such as re-entry and efference copy/collateral discharge, which are relevant to schizophrenia, can also be studied in very concrete ways in the visual cortex. Studies of all of these phenomena in schizophrenia have found abnormalities suggesting that basic modulatory processes are impaired at the perceptual level, and that this may be the operative factor in "high-level" impairments such as those involving working memory. In support of this is a recent study on cognitive control in schizophrenia that attributed its failure to impaired contextual modulation in the frontal cortex (Barbalat et al., 2009).

Although not noted in the SRF summary of the Dias paper, an important point relevant to their view that sensory and perceptual abnormalities are primary disturbances in schizophrenia is (as noted in the Introductory paper to the forthcoming Schizophrenia Bulletin theme section on Vision Science and Schizophrenia Research): there is also much evidence that 1) many aspects of perception do not require cognitive control; and 2) there are disturbances in perceptual processing in schizophrenia that cannot be explained in terms of higher-level deficits. Regarding the first point, for example, intact contour integration has been observed in the neglected visual field in patients with hemispatial neglect, and in healthy subjects, contour integration happens automatically even when participants do not want it to or are not attending to the contour-containing stimuli. Moreover, visual processes such as spatial frequency processing, perceptual organization, form perception, and motion perception are found throughout the animal kingdom in species with far less frontal cortex development than humans. Regarding the second point, involvement of the occipital lobe in contour integration has been found in anesthetized monkeys, with the same regions showing hypoactivation in studies with schizophrenia patients, suggesting that the basic binding process that is impaired does not normally require cognitive control. Similarly, contour integration deficits have been repeatedly demonstrated in people with amblyopia, a condition involving reduced integration of information in early visual cortex regions, associated with suppressed input from one eye, and these patients are not characterized by impairments in frontal cortex functioning.

Further, as demonstrated by Kéri and colleagues (Kéri et al., 2009), in collinear facilitation tasks, when both schizophrenia patients and controls attend to flankers to the same degree (as verified by a secondary task), only patients show impaired ability at integrating the flanker elements. Patients are also less susceptible to certain illusions, and it is doubtful that deficits in cognitive control help subjects recover correct shape information. In addition, there is abundant evidence for magnocellular pathway deficits in schizophrenia, based on studies of motion, spatial frequency, and contrast sensitivity processing. These deficits have well-known origins in early sensory processing centers. Finally, findings of increased occipital lobe gray matter loss in poor outcome patients (see Mitelman and Buchsbaum, 2007), the subgroup that typically shows the most severe perceptual impairments, suggests a primary role for occipital lobe dysfunction in visual stimulus assembly failures in schizophrenia.

In short, the evidence from vision research highlights that much coordination of perceptual and cognitive activity emerges via self-organization in local populations of neurons, which is a general property of circuitry throughout the brain. This view does not negate the importance of different brain regions for specialized processing, such as that the occipital lobe is a visual information processor, whereas the prefrontal cortex is heavily involved in strategic planning and goal maintenance. Nor does it negate the possibility that there are abnormalities in schizophrenia that are due to faulty interregional interactions. Rather, this view highlights the primary importance of coordinating processes for diverse mental functions, the likelihood that at least some perceptual phenomena and their abnormalities in schizophrenia are best accounted for by failures in this process within visual pathways, and the relative simplicity of viewing and comprehending coordinating processes, and their impairments in schizophrenia, through the lens of vision research.

Finally, a minor correction is necessary to the statement in the SRF summary that studies have not found hypoactivation in visual cortex regions in schizophrenia. In fact, this has been found, in studies of both visual processing (e.g., Silverstein et al., 2009) and working memory (the study by Haenschel et al. noted above). Moreover, in some cases, as with face processing, compensatory activity in higher regions has been observed in the context of reduced activity in striate and/or extrastriate visual cortex regions (e.g., Silverstein et al., 2009; Silverstein et al., 2010).

In short, there is compelling evidence for visual processing dysfunction in schizophrenia (along with impairments in other sensory domains), and these impairments serve as rate limiting factors for later processing. They also demonstrate, in concrete form, much about what is altered throughout the brain in schizophrenia. Greater attention to these issues can have the effect of refining our view of schizophrenia as a whole brain disorder, and moving away from the view that a primary failure in executive control (of presumably normal representations) defines what schizophrenia is.


Haenschel C, Bittner RA, Haertling F, Rotarska-Jagiela A, Maurer K, Singer W, Linden DE. Contribution of impaired early-stage visual processing to working memory dysfunction in adolescents with schizophrenia: a study with event-related potentials and functional magnetic resonance imaging. Arch Gen Psychiatry . 2007 Nov 1 ; 64(11):1229-40. Abstract

Silverstein SM, Bakshi S, Nuernberger S, Carpinello K, Wilkniss S. Effects of stimulus structure and target-distracter similarity on the development of visual memory representations in schizophrenia. Cogn Neuropsychiatry . 2005 Jun 1 ; 10(3):215-29. Abstract

Gold JM, Hahn B, Zhang WW, Robinson BM, Kappenman ES, Beck VM, Luck SJ. Reduced capacity but spared precision and maintenance of working memory representations in schizophrenia. Arch Gen Psychiatry . 2010 Jun 1 ; 67(6):570-7. Abstract

Barbalat G, Chambon V, Franck N, Koechlin E, Farrer C. Organization of cognitive control within the lateral prefrontal cortex in schizophrenia. Arch Gen Psychiatry . 2009 Apr 1 ; 66(4):377-86. Abstract

Kéri S, Kelemen O, Benedek G. Attentional modulation of perceptual organisation in schizophrenia. Cogn Neuropsychiatry . 2009 Mar 1 ; 14(2):77-86. Abstract

Mitelman SA, Buchsbaum MS. Very poor outcome schizophrenia: clinical and neuroimaging aspects. Int Rev Psychiatry . 2007 Aug 1 ; 19(4):345-57. Abstract

Silverstein SM, Berten S, Essex B, Kovács I, Susmaras T, Little DM. An fMRI examination of visual integration in schizophrenia. J Integr Neurosci . 2009 Jun 1 ; 8(2):175-202. Abstract

Silverstein SM, All SD, Kasi R, Berten S, Essex B, Lathrop KL, Little DM. Increased fusiform area activation in schizophrenia during processing of spatial frequency-degraded faces, as revealed by fMRI. Psychol Med . 2010 Jul 1 ; 40(7):1159-69. Abstract

View all comments by Steven Silverstein

Related News: Poor Working Memory Starts Early in Processing Pipeline

Comment by:  Michael F. Green, SRF Advisor
Submitted 30 June 2011
Posted 30 June 2011

The paper by Dias et al. makes an important contribution to our understanding of perceptual processes in schizophrenia. Steve Silverstein has already provided a thoughtful and detailed comment on it. I have just three points to add:

First, the authors provide an excellent demonstration of the explanatory value of bottom-up perceptual processing for vigilance and working memory tasks, such as the AX version of the continuous performance task. However, the implications extend beyond such tasks. There is increasing support for the importance of early perceptual processes (both auditory and visual) for social cognition, including prosody detection and social perception (Leitman et al., 2005; Wynn et al., 2010). In addition, early visual processing has been part of outcome models that reach to community functioning (Rassovsky et al., 2011; Butler et al., 2005). Therefore, the value of perceptual models is that they influence how we interpret cognitive tasks that have typically been viewed in terms of top-down control processes (as in Dias et al.). They also provide a way to map the steps from brain processes to daily functioning.

Second, when it comes to mapping the stages of processing, EEG is still king. Although functional magnetic resonance imaging (fMRI) is increasingly used to examine briefly presented stimuli, there is still no better way than EEG to separate the cortical response to one stimulus versus another, or to separate stages of processing. This is particularly true when one is trying to distinguish among stages of perceptual processing, or between sensory and cognitive event-related potentials.

Third, it should be noted that a bottom-up focus (as described in Dias et al.) does not necessarily implicate the earliest stages of perceptual processes. There are several examples in which patients and controls differ at early, but not the earliest, processing stages. Nor do the earliest stages always account for variance in later stages. For example, we have consistently found visual backward masking deficits in schizophrenia after matching subjects on their accuracy for detecting an unmasked target. In other words, the ability to detect a briefly presented target alone does not explain patients’ problems in processing visual stimuli presented in rapid succession (Green et al., 2003). Also, we have found that fMRI activation in primary visual areas (i.e., retinotopic areas) is intact in schizophrenia, but that differences emerge at slightly later visual stages, such as object perception, which is conducted by the lateral occipital complex (Green et al., 2009).

Lastly, we have found that ERP (event-related potential) components of emotional responses to evocative pictures are intact for the first few hundred milliseconds following stimulus onset (Horan et al., 2010).

The implication is that bottom-up models have terrific explanatory value for higher-level cognitive processes, social cognition, and even community functioning. At the same time, they have their limits and present their own mysteries, such as why bottom-up does not always seem to start at the bottom.


Leitman DI, Foxe JJ, Butler PD, Saperstein A, Revheim N, Javitt DC. Sensory contributions to impaired prosodic processing in schizophrenia. Biol Psychiatry. 2005;58(1):56-61. Abstract

Wynn JK, Sugar C, Horan WP, Kern R, Green MF. Mismatch negativity, social cognition, and functioning in schizophrenia patients. Biol Psychiatry. 2010;67(10):940-7. Abstract

Rassovsky Y, Horan WP, Lee J, Sergi MJ, Green MF. Pathways between early visual processing and functional outcome in schizophrenia. Psychol Med. 2011;41:487-97. Abstract

Butler PD, Zemon V, Schechter I, Saperstein AM, Hoptman MJ, Lim KO, Revheim N, Silipo G, Javitt DC. Early-stage visual processing and cortical amplification deficits in schizophrenia. Arch Gen Psychiatry. 2005;62(5):495-504. Abstract

Green MF, Nuechterlein KH, Breitmeyer B, Tsuang J, Mintz J. Forward and backward visual masking in schizophrenia: influence of age. Psychol Med. 2003;33:887-95. Abstract

Green MF, Lee J, Cohen MS, Engel SA, Korb AS, Nuechterlein KH, Wynn JK, Glahn DC. Functional neuroanatomy of visual masking deficits in schizophrenia. Arch Gen Psychiatry. 2009;66(12):1295-1303. Abstract

Horan WP, Wynn JK, Kring AM, Simons RF, Green MF. Electrophysiological correlates of emotional responding in schizophrenia. J Abnorm Psychol. 2010;119(1):18-30. Abstract

View all comments by Michael F. Green