| The Neurobiology of the Arts |
Light Vision The answer may lie in part with the painting’s luminance, or perceived lightness. The elements of visual art have long been held to be color, shape, texture, and line. But an even more basic distinction lies between color and luminance. Color can convey emotion and symbolism, but luminance alone defines shape, texture, and line. “Colors are only symbols,” Pablo Picasso once wrote. “Reality is to be found in lightness alone.” Most people are comfortable talking about color. Yet luminance, even though it is more fundamental, is dimly understood. Given two patches of gray, it is easy to identify which is lighter, but given two colors, it is often difficult to draw such a distinction. A monochromatic rendering of Impression, Sunrise reveals that Monet painted the sun at exactly the same luminance as the gray of the clouds. If he had rendered it in a strictly representational style, the sun would have been brighter than the sky by a factor too large to have been duplicated with pigments. If he had made the sun lighter—which is closer to the way it would appear in reality—it would have lost its quavering luminosity and would have seemed, paradoxically, less bright. Rather than appearing as a source of light, the sun would have looked like a cutout affixed to the clouds. By rendering the sun the exact luminance as the sky, Monet achieved an eerie effect: his orange sun appears to pulsate across the grayish-green water. Gray Matters Color and luminance play distinct roles in our perception of art—and even of real life—because our visual systems analyze color and luminance separately. The areas of our brain that process information about color, in the temporal lobe, are several centimeters away from the areas that analyze luminance, in the parietal lobe. They are as anatomically distinct as vision is from hearing. The luminance system, which is evolutionarily older, is common to all mammals; the parts of the brain that process color information are present only in primates. That is probably why the most primitive visual information about a scene is found in variations of luminance. It does not matter which color is used to convey the luminance signal, because the parts of our brains that analyze the most basic features of a scene are, quite literally, colorblind. On a gross level, the visual system is a single pathway in the brain. On a finer scale, however, this pathway consists of two major subdivisions. The evolutionarily older large-cell subdivision is responsible for our perception of motion, space, position, depth, figure-and-ground segregation, and the overall organization of the visual scene. This subdivision is called the “Where” system. The small-cell subdivision, which is well developed only in primates, is responsible for our ability to recognize objects, including faces, in color and in complex detail. This newer system is called the “What” system. The Where and What systems differ not only in the kind of information they extract from the environment, but also in how they process light signals. The Where system is colorblind; the What system carries information about color. The Where system has a much higher sensitivity to small differences in brightness. It is also faster and more transient in its responses and has a slightly lower acuity, or resolution. In the retina, thalamus, and early cortical areas, the Where and What systems are physically interdigitated, yet they keep the information they process largely separate. At higher levels, the two subdivisions become even more spatially segregated. Evolution likely accounts for these subdivided visual tasks. The Where system in humans and other primates resembles the entire visual system of lower mammals. These animals are much less sensitive to color than we are, and they can neither scrutinize objects nor accurately discriminate them on the basis of visual attributes. Instead they are sensitive to objects in motion, because things that move—whether prey or predator—are likely to be important. Also, because the primitive visual system must have been used to navigate through three-dimensional environments, it had to have been able to process depth information and distinguish objects from their backgrounds. As the more complicated primate visual system evolved, the original system was maintained, probably because it was simpler to overlay color vision and object recognition onto the existing system than it would have been to integrate the two. Artistic License In the first and most fundamental step of our visual processing, our retinal ganglion cells are excited by light impinging on their receptive field centers. Notably, however, they are inhibited by light falling on the immediately surrounding region. The net effect is to record the relationship of “center” to “surround.” Cells at the next stage of processing, in the thalamus, show a similar center/surround organization, which makes cells at these early stages of the visual system sensitive to discontinuities in the pattern of light falling on the retina rather than to the absolute level of light. Neurons respond best to sharp changes, rather than to gradual shifts in luminance. This wiring allows the visual system to ignore gradual changes in light and the overall level of the illuminant, factors that are usually not important biologically. Many visual modalities—such as luminance, color, motion, and depth—exhibit greater sensitivity to abrupt than to gradual change. In each modality, this selectivity is due to an underlying center/surround organization. It makes adaptive sense for our visual system to be designed in this way because it is more efficient to encode only those parts of the image that have changes or discontinuities than to encode the entire image. The visual system in a sense compresses images because it takes energy for nerve cells to signal; the fewer cells that signal, the more energy is conserved. Higher-level visual processing, such as object recognition, is essentially the end result of extracting the information content of an image. Artists can take advantage of this quirk in our visual system to expand the apparent range of reflectances of paints. Although a real scene may contain a large spectrum of luminances, our visual system initially analyzes each part of the scene separately. So by introducing gradual changes in the background luminance, for example, an artist can shift the apparent luminance of the foreground in the opposite direction. Tricks of the Light Artists have been playing with luminance for centuries. In his 1632 painting Meditating Philosopher, Rembrandt used variations in luminance to create an almost ethereal golden glow. If this were a real scene, the luminance of the window would likely be hundreds of times that of the upper reaches of the shadowy staircase—an effect nearly impossible to duplicate with paint alone. The paint representing the window actually reflects only 15 times more light than the paint representing the shadows in the lower left corner of the painting, but we perceive the window section to be substantially lighter. Rembrandt created another illusion by painting the philosopher’s head on a darker background and the crosspiece of the window frame on a lighter background. The head thus appears relatively light and the window frame relatively dark, even though the head is darker than the frame. We cannot easily perceive the differences in the backgrounds because they meld gradually into one another. By using a combination of gradual background changes and local abrupt changes in luminance, Rembrandt simulated a much larger range of luminances than his pigments could supply. Over the centuries, artists continued to increase their command of luminance to enhance their ability to represent depth on a two-dimensional canvas. This trend toward representationalism reached a pinnacle in the early nineteenth century with the work of Jean-Auguste-Dominique Ingres, whose paintings have an amazingly photographic quality. Art historians have suggested that Ingres must have used a camera lucida or other optical aid to project an image of the scene onto the canvas or drawing tablet, so uncannily does he capture the gradations of luminance in his subjects. Then, toward the end of the nineteenth century, the Impressionists aligned themselves against the representational style of art epitomized in the work of Ingres. Some experimented with color and luminance, sometimes using unrealistic color gradations or abandoning luminance differences entirely. Still Lifes in Motion One of the Impressionists’ most novel accomplishments is the shimmering, alive quality they achieved in many of their paintings. The sensation of movement in Impression, Sunrise—and some of Louis Leroy’s disdain for the painting—stemmed in part from Monet’s use of quick dabs of paint, which required the viewer’s eye to blend the colors. “Wallpaper in its original state is more finished than this seascape!” Leroy groused. And yet it is clear that some of the color combinations the Impressionists used have so little luminance contrast that they create the illusion of motion. We perceive illusory motion in images made from equiluminant colors for the same reason we don’t see appropriate depth in these images: our Where system can’t distinguish between equiluminant colors. Therefore if an image is composed of equiluminant colors, our What system can see those objects, but our Where system—which is responsible for our ability to see motion and position, as well as depth—cannot register their position and stability, so they can seem to jitter. Monet’s The Poppy Field Outside of Argenteuil is a good example of this illusion. The red of the flowers is nearly equiluminant with the green of the grass and the skirt of the woman in the foreground. Our color-selective What system can easily distinguish the poppies and the skirt from the grass. But the colors, although bright, do not have enough luminance contrast for our Where system to see them. Their position seems uncertain, giving them an illusory instability. They can seem to move, as if stirred by a breeze. Our eyes can be similarly tricked by repetitive high-contrast lines, which tend to create motion perpendicular to their own orientation. Light shining through horizontal venetian blinds, for example, will induce the appearance of vertical motion on an adjacent wall, a phenomenon known as the McKay illusion. An extreme example of this illusion is Isia Leviant’s Enigma. The juxtaposition of luminance-contrast borders with areas of equiluminance can cause the illusion of motion; after looking at Enigma for a minute or so, the viewer should notice a streaming effect in the colored circles. The streaming always moves perpendicularly to the high-contrast lines, which induce it. We do not yet understand why a large field of high-contrast lines induces an illusion of motion. Some of Monet’s paintings likely induce a mild form of this deception to help create their illusory sense of movement.  Art Mystery Five hundred years after Mona Lisa sat for Leonardo da Vinci, we’re still trying to understand what makes her painted image so lifelike. She seems to smile until you look at her mouth, then her smile fades, like a dim star that disappears as soon as you gaze directly at it. One popular idea is that Leonardo used sfumato—a technique of subtly blurring sharp outlines—to make her expression ambiguous. That hypothesis would mean that her smile would vary depending on the viewer’s imagination or state of mind, but its variability is more systematic than that. While looking at the painting one day, I noticed that Mona Lisa’s expression changed according to how far the center of my gaze strayed from her mouth. These systematic transformations suggested that her lifelike quality was not so mysterious after all. Her smile, I realized, is differentially apparent in different parts of our visual field. To understand how Mona Lisa’s smile would look at a range of eccentricities, I processed images of her face to reveal its fine, medium, and coarse components. A clear smile is more apparent in the coarse and medium components of the images than in the fine detail image. This means that if the center of your gaze falls on the background or on Mona Lisa’s hands, her mouth—which is then seen by your peripheral, low-resolution vision—appears cheerful. When you look directly at her mouth, your high-resolution foveal vision sees details that take away the grin. This explains the elusive quality of her expression: you literally can’t catch her smile by looking at her mouth. The spatial imprecision of our peripheral vision has interesting implications for our perception of some Impressionist paintings, too. In Monet’s Rue Montorgueil in Paris, Festival of June 30, 1878, for example, details are spatially jumbled. If you look carefully at the flags just to the left or right of the center of Rue Montorgueil, you can see that the blue, white, and red brushstrokes, representing the stripes of the tricolored flags of France, are not always well aligned or even adjacent to one another. This spatial imprecision differs from a simple blurring: it mimics the spatial imprecision in our peripheral visual field. Our peripheral vision occasionally makes erroneous correlations between objects seen and objects known to exist. This phenomenon, called illusory conjunction, occurs when items are presented either peripherally or transiently. The flags along the Rue Montorgueil look fine when you first glance at the painting, but not if you look directly at them, or after you study those parts specifically. The painting’s spatial imprecision is not immediately noticeable because our own spatial imprecision allows illusory conjunctions to complete the objects. That explains why we see complete flags, even though many of them are just single strokes of paint. Low spatial precision can lend vitality to a painting, because our visual system fills in the picture differently with each glance. It also gives the painting a transient feel because such imprecision is compatible with a single glance, a fleeting moment in time. Because of the low spatial resolution of peripheral vision, we cannot take in a detailed percept of the entire scene in a single glance; we see clearly only the part of the scene that our center of gaze happens to light on. “The visual sensation that imprints itself on the retina lasts but a second, or even less,” wrote Impressionist painter Gustave Caillebotte, a master of the art of capturing a fleeting moment. “That’s the impression that we had to pursue.” By comparison, Nicolas Poussin’s highly detailed, action-packed Rape of the Sabine Women looks relatively static, because we can see hundreds of details. Seeing so many details is incompatible with the transience of the incident depicted—by the time our eyes move from one act of savagery to another, the scene should have changed. The longer you look, the colder and more frozen the figures in the painting seem. In the Shade When a light source illuminates a three-dimensional object, different parts of the object’s surface reflect different amounts of light, depending on the angle of the light hitting them. We see these differences as changes in luminance, or shading, which is another depth cue that, like perspective, artists must learn to render. To use shading effectively, artists have to surmount several challenges. They must learn to see luminance gradation and to evaluate luminance independent of color. Even then, they often find it impossible to duplicate those luminance ranges with pigments because of the limited range of reflectances available even with the best paints. The range of luminances in a given scene is almost always far greater than the array of values an artist can achieve using pigments. Inside a typical room, for example, luminances vary widely: a light source, such as a window or lamp, might be hundreds of times brighter than the shadowed region under a desk. The luminance in outdoor scenes usually varies by a factor of a thousand. We know that luminance contrast, not color, is necessary for depth perception. A corollary of this principle is that, as long as you have the appropriate luminance contrast, you can use any hue you want and still portray a shape in three dimensions with shading. In Henri Matisse’s La Femme au Chapeau, for example, the shadows and most of the planes of the subject’s face are peculiar colors. Although it is difficult to imagine what kind of lighting would cast blue and mauve shadows, the three-dimensional shape of the woman’s face does not seem unnatural because the patches of bizarre colors have the correct relative luminance to represent planes and shadows. Matisse himself explained, “While following the impression produced on me by a face, I have tried not to stray from the anatomical structure.” Matisse had discovered that he could use any hue and still portray the three-dimensional shape he wanted as long as the luminance was appropriate. The art collector Leo Stein, who eventually bought the painting, wrote, “It was a tremendous effort on his part, a thing brilliant and powerful, but the nastiest smear of paint I had ever seen.” A Double Take Although late Renaissance painters attained a photographically realistic use of perspective and shading, those techniques alone could not convey an authentic feeling of three-dimensionality. No matter how convincingly an artist renders shading and perspective, two other important cues—stereopsis and relative motion—inform the viewer’s brain that the painting is, in fact, flat. Since our two eyes view the world from slightly different positions, the images on the two retinas differ slightly. Stereopsis is the ability of our visual system to interpret the disparity between the two images as depth. A stereoscope, a device popular in the mid-nineteenth century, presented two slightly different pictures, one to each eye, to give a vivid sense of depth. The View-Masters many of us enjoyed as children also work on this principle, showing three-dimensional images of pterodactyls, volcanoes, and Donald Duck. The same part of the brain that codes stereopsis codes depth from relative motion, so movements as small as the distance between our eyes are large enough to produce a strong depth signal. We glean information about distance from the relative motion of objects as we move past them. When you walk down a street at night, for example, the objects close to you, such as the trees along the sidewalk, seem to pass more quickly than the houses or trees farther away. Those at even greater distances, such as the moon, seem stationary. We also pick up relative movement cues from the small head motions we make even when we stand still in front of a painting. No matter how skillfully the artist conveys depth through the use of perspective and shading, because the images in our two eyes are identical and because there is no relative movement between objects in the painting, our brains register the painting as flat. The Impressionists found multiple ways to trick our brains, though. In most Impressionist paintings, cues such as perspective or shading, rendered in luminance contrast, convey a sense of depth. The blurriness and deliberate lack of details characteristic of many Impressionist paintings also contribute to a sense of three-dimensionality. To see stereoscopic depth, the image needs to be detailed enough to allow us to detect the slight differences between our two eyes’ images. By eliminating some spatial details and blurring others, an artist can hinder stereopsis from revealing the flatness of the image. This allows other depth cues in the painting, such as shading and perspective, to produce a more powerful signal because they are not as strongly contradicted by stereopsis. The notable ability of some Impressionist and Post-Impressionist paintings to invoke an illusion of depth, or a sensation of atmosphere, also likely arises from the rendering of semiregular patterns of leaves or flowers, or even from coarse brush strokes. Rue Montorgueil, for example, produces an illusion of depth because of the semirepetitive patterns of the flags. Ironically, this effect goes beyond what realism could achieve—short of making two slightly different paintings and using stereo viewers—to generate a sense of depth. The sense of atmosphere is particularly striking in Pierre-Auguste Renoir’s A Girl Gathering Flowers. The dabs of paint can be mismatched in the images in our two eyes, giving the painting an illusory sense of a three-dimensional volume filled with small floating elements, such as flower petals, insects, and pollen. Vision Quest The ways in which we process color and luminance hold ramifications for more than paintings; they also affect our perceptions of television, computer graphics, photography, color printing, and movies. These technologies are all flat, like painting, so they use the same kinds of cues—perspective, shading, and occlusion—to give an illusory sense of depth. They also have the same problem as paintings in that our stereopsis registers the images as flat. But movies and television have the potential for a powerful additional depth cue—relative motion. If you close one eye and gaze steadily at, say, the edge of this magazine, you may find that it does not seem clearly in front of background objects. But by moving your head slightly from side to side you can make it jump back into proper apparent depth. That is because relative motion of objects at different distances is a strong cue to their distance from the observer. Relative motion of objects in movies and television can be a powerful cue to depth and can even induce an illusion of being propelled through space. Who didn’t have to grip their seat the first time they saw the opening credits for Star Wars? Recent advances in our understanding of the human visual system allow us to look at art—and our perceptions of the world—in new ways. Without understanding the underlying neurobiology of color and luminance recognition, artists, advertisers, psychologists, and the technology industry have discovered various phenomena that turn out to be based on the parallel organization of our visual systems. It will be interesting to see whether an explicit understanding of the neurobiology of vision will lead to more sophisticated effects and illusions and a greater knowledge of brain function in general. Margaret Livingstone, PhD, is a professor of neurobiology at Harvard Medical School. This article was largely adapted from her book Vision and Art: The Biology of Seeing, published by Henry N. Abrams, Inc., in 2002. This article appeared in the Autumn 2003 issue of the Harvard Medical Alumni Bulletin. Photo captions: Louis Leroy found Monet’s Impression, Sunrise “vague and brutal” and “worse than anyone hitherto had dared to paint” (top photo); Mona Lisa’s expression changes depending on how far the viewer’s center of gaze is from her mouth, as seen in the photo montage in the middle of the story. A clear smile is more evident on her face in details that show the coarse and medium image components (left and center) than in one (right) that shows only fine details. Photos: Eric Lessing/Art Resource, New York (Monet painting); Margaret Livingstone (Mona Lisa montage) |
|