Study finds prominent link between sound and sight

Involuntary attention to sound influences visual perception.

SAN DIEGO — The involuntary orienting of attention to sound enhances early perceptual processing of visual stimuli, according to a study that evaluated the effect of hearing on people’s sight.

“We found a strong association between sight and sound,” said John J. McDonald, PhD, a post-doctoral research fellow in neurosciences here at the University of California at San Diego and first author of the study. “Attending to the location of the sound gave a significant boost in the subjects’ visual perception.”

The study, which recently appeared in Nature, used a signal detection measure of perceptual sensitivity. “This is a more precise measure of a person’s ability to see the stimulus,” Dr. McDonald told Ocular Surgery News. “By using the signal-detection theory, we found that orienting attention to a sound influenced ability to perceive a light.”

Two experiments

The researchers conducted two experiments, “to replicate the major findings,” Dr. McDonald said. A total of 15 paid volunteers (ages 19 to 30) participated in experiment one and 18 paid volunteers (ages 19 to 37) took part in experiment two. The setting was a large sound-attenuation chamber. “Subjects looked straight ahead, and without moving their eyes, they tried to detect a faint green light that appeared in their peripheral vision, either to the left or right,” Dr. McDonald said. “The green light, when present, was obscured by a much brighter red light, which occurred immediately afterward.”

Before the green and red lights were introduced, a loud but irrelevant noise was presented to the left or right side. “The sound was presented between 100 ms and 300 ms before the green light, and it lasted for about 85 ms,” Dr. McDonald said. “This sound was a warning event that oriented subjects to the location of the sound before the lights appeared. But the sound did not indicate whether the lights would occur on the left or right, or whether the green light would be present. There was no reason for the subjects to pay attention to the sound at all. In fact, we told subjects to ignore the sound and only to judge whether the green light occurred.” Still, subjects were better able to see the green light when it occurred on the same side as the sound than when it occurred on the opposite side.

Three different measures of perceptual processing were evaluated: reaction time, response accuracy and perceptual sensitivity. “In the past, most researchers, including myself, have relied only on response time, which is a rather crude measure of perceptual processing,” Dr. McDonald said. “We have previously found that attending to a sound facilitates responses to visual targets. We’ve tried to infer from those findings that somehow the sound influences a person’s vision. But we really can’t make that inference based on a subject’s speed of response because that is associated with many things, like decision and motoric processes,” he said.

Enhanced vision near sound

The present study used more precise measures of visual perception, including response accuracy and a signal-detection measure of perceptual sensitivity called d-prime.

“We found that people detected the green light more quickly when it occurred on the same side of fixation as the previous sound,” Dr. McDonald said. “We also found that people detected the green light more accurately when it occurred on the same side as the previous sound.” Most important, though, the study found perceptual sensitivity was significantly higher when the light and sound occurred on the same side. According to Dr. McDonald, the sound boosted perceptual sensitivity by about 8% for lights that appeared on the same side.

“I’m excited about relating the processing of lights and sound, rather than to simply study either sight or hearing independently,” Dr. McDonald said. “Our study provides psychophysical evidence that a sudden sound improves the detectability of a subsequent flash appearing at the same location.”

One immediate implication of the study is that it may lead to to better treatments for individuals with attention deficits. “This includes attention-deficit disorder and patients with schizophrenia who have problems attending to environmental events, particularly those who have problems associating stimuli from different modalities,” Dr. McDonald said.

A second application may be in designing better warning systems or human-computer interaction devices.

“The air traffic control radar screens are predominantly visual instruments,” Dr. McDonald said. “It would be helpful if just prior to the appearance of a new light (new plane), a sound was presented in the same general area to attract the controller’s attention to that area of the radar screen. Traffic controllers could preprocess that area and have heightened perception, so they would be less likely to miss crucial information.”

The researchers are currently following up their published study with brain activity measures to investigate how orienting attention to sound influences visual processing.

For Your Information:

• John J. McDonald, PhD, can be reached at the Department of Neurosciences, University of California-San Diego, 9500 Gilman Dr., Mail Code 0608; La Jolla, CA 92093-0608; (858) 534-5560; fax: (858) 534-5562; e-mail: jmcd@sfu.ca.

Reference:

• McDonald JJ, Teder-Salejarvi WA, Hillyard SA; Involuntary orienting to sound improves visual perception. Nature. 2000:407:906-908.