Central processing of visual information

Vivid images of the world, with detail, colour, and meaning, impinge on human consciousness. Many people believe that humans simply see what is around them. However, internal images are the product of an extraordinary amount of processing, involving roughly half the cortex (the convoluted outer layer) of the brain. This processing does not follow a simple unitary pathway. It is known both from electrical recordings and from the study of patients with localized brain damage that different parts of the cerebral cortex abstract different features of the image; colour, depth, motion, and object identity all have “modules” of cortex devoted to them. What is less clear is how multiple processing modules assemble this information into a single image. It may be that there is no resynthesis, and what humans “see” is simply the product of the working of the whole visual brain.

The axons of the ganglion cells leave the retina in the two optic nerves, which extend to the two lateral geniculate nuclei (LGN) in the thalamus. The LGN act as way stations on the pathway to the primary visual cortex, in the occipital (rear) area of the cerebral cortex. Some axons also go to the superior colliculus, a paired formation on the roof of the midbrain. Between the eyes and the lateral geniculate nuclei, the two optic nerves split and reunite in the optic chiasm, where axons from the left half of the field of view of both eyes join. From the chiasm the axons from the left halves make their way to the right LGN, and the axons from the right halves make their way to the left LGN. The significance of this crossing-over is that the two images of the same part of the scene, viewed by the left and right eyes, are brought together. The images are then compared in the cortex, where differences between them can be reinterpreted as depth in the scene. In addition, the optic nerve fibres have small, generally circular receptive fields with a concentric “on”-centre/“off”-surround or “off”-centre/“on”-surround structure. This organization allows them to detect local contrast in the image. The cells of the LGN, to which the optic nerve axons connect via synapses (electrical junctions between neurons), have a similar concentric receptive field structure. A feature of the LGN that seems puzzling is that only about 20–25 percent of the axons reaching them come from the retina. The remaining 75–80 percent descend from the cortex or come from other parts of the brain. Some scientists suspect that the function of these feedback pathways may be to direct attention to particular objects in the visual field, but this has not been proved.

The LGN in humans contain six layers of cells. Two of these layers contain large cells (the magnocellular [M] layers), and the remaining four layers contain small cells (the parvocellular [P] layers). This division reflects a difference in the types of ganglion cells that supply the M and P layers. The M layers receive their input from so-called Y-cells, which have fast responses, relatively poor resolution, and weak or absent responses to colour. The P layers receive input from X-cells, which have slow responses but provide fine-grain resolution and have strong colour responses. The division into an M pathway, concerned principally with guiding action, and a P pathway, concerned with the identities of objects, is believed to be preserved through the various stages of cortical processing.

The LGN send their axons exclusively to the primary visual area (V1) in the occipital lobe of the cortex. The V1 contains six layers, each of which has a distinct function. Axons from the LGN terminate primarily in layers four and six. In addition, cells from V1 layer four feed other layers of the visual cortex. American biologist David Hunter Hubel and Swedish biologist Torsten Nils Wiesel discovered in pioneering experiments beginning in the late 1950s that a number of major transformations occur as cells from one layer feed into other layers. Most V1 neurons respond best to short lines and edges running in a particular direction in the visual field. This is different from the concentric arrangement of the LGN receptive fields and comes about by the selection of LGN inputs with similar properties that lie along lines in the image. For example, V1 cells with LGN inputs of the “on”-centre/“off”-surround type respond best to a bright central stripe with a dark surround. Other combinations of input from the LGN cells produce different variations of line and edge configuration. Cells with the same preferred orientation are grouped in columns that extend through the depth of the cortex. The columns are grouped around a central point, similar to the spokes of a wheel, and preferred orientation changes systematically around each hub. Within a column the responses of the cells vary in complexity. For example, simple cells respond to an appropriately oriented edge or line at a specific location, whereas complex cells prefer a moving edge but are relatively insensitive to the exact position of the edge in their larger receptive fields.

Each circular set of orientation columns represents a point in the image, and these points are laid out across the cortex in a map that corresponds to the layout in the retina (retinotopic mapping). However, the cortical map is distorted compared with the retina, with a disproportionately large area devoted to the fovea and its immediate vicinity. There are two retinotopic mappings—one for each eye. This is because the two eyes are represented separately across the cortex in a series of “ocular dominance columns,” whose surfaces appear as curving stripes across the cortex. In addition, colour is carried not by the orientation column system but by a system prosaically known as “blobs.” These are small circular patches in the centre of each set of orientation columns, and their cells respond to differences in colour within their receptive fields; they do not respond to lines or edges.

The processing that occurs in area V1 enables further analysis of different aspects of the image. There are at least 20 areas of cortex that receive input directly or indirectly from V1, and each of these has a retinotopic mapping. In front of V1 is V2, which contains large numbers of cells sensitive to the same features in each eye. However, within V2 there are small horizontal differences in direction between the eyes. Separation of the images in the two eyes results from the presence of objects in different depth planes, and it can be assumed that V2 provides a major input to the third dimension in the perceived world. Two other visual areas that have received attention are V4 and MT (middle temporal area, or V5). British neurobiologist Semir Zeki showed that V4 has a high proportion of cells that respond to colour in a manner that is independent of the type of illumination (colour constancy). This is in contrast to the cells of V1, which are responsive to the actual wavelengths present. In rare instances when V4 is damaged, the affected individual develops central achromatopsia, the inability to see or even imagine colours despite a normal trichromatic retina. Thus, it appears that V4 is where perceived colour originates. MT has been called the motion area, and cells respond in a variety of ways not only to movements of objects but also to the motion of whole areas of the visual field. When this area is damaged, the afflicted person can no longer distinguish between moving and stationary objects; the world is viewed as a series of “stills,” and the coordination of everyday activities that involve motion becomes difficult.

In the 1980s American cognitive scientists Leslie G. Ungerleider and Mortimer Mishkin formulated the idea that there are two processing streams emanating from V1—a dorsal stream leading to the visual cortex of the parietal lobe and a ventral stream leading to the visual regions of the temporal lobe. The dorsal stream provides the parietal lobe with the position and information needed for the formulation of action; MT is an important part of this stream. The ventral stream is more concerned with detail, colour, and form and involves information from V4 and other areas. In the temporal lobe there are neurons with a wide variety of spatial form requirements, but these generally do not correspond exactly with any particular object. However, in a specific region of the anterior part of the inferotemporal cortex (near the end of the ventral stream) are neurons that respond to faces and very little else. Damage to areas near this part of the cortex can lead to prosopagnosia, the inability to recognize by sight people who are known to the subject. Loss of visual recognition suggests that information supplied via the ventral stream to the temporal lobe is refined and classified to the point where structures as complex as faces can be represented and recalled.

Great progress has been made over the last century in understanding the ways that the eye and brain transduce and analyze the visual world. However, little is known about the relationship between the objective features of an image and an individual’s subjective interpretation of the image. Scientists suspect that subjective experience is a product of the processing that occurs in the various brain modules contributing to the analysis of the image.