Presentation Information
[P1-38]Understanding Discomfort Caused by Audiovisual Temporal Asynchrony: Insights from Egg Cracking and Grissini Breaking Videos
*Mayuka Hayashi1, Waka Fujisaki1 (1. Japan Women’s Univ. (Japan))
Keywords:
Audiovisual temporal asynchrony,simultaneity judgment,discomfort judgment,temporal window,psychophysical experiment
This study investigated how a sense of discomfort may be triggered by a time lag between visual and auditory stimuli under conditions that resemble everyday experiences. We conducted a psychophysical experiment using videos in which audio-visual time lags were manipulated across seven levels (±0 ms, ±133 ms, ±266 ms, ±400 ms). Participants (N = 15) were asked to judge whether the auditory and visual stimuli were simultaneous (simultaneity judgment) and whether they felt discomfort (discomfort judgment), using a two-alternative forced choice (2 AFC) method. The stimuli featured two everyday actions: cracking an egg and breaking a breadstick (grissini).
We proposed three hypotheses to explain the emergence of discomfort. Hypothesis 1 suggested that discomfort and simultaneity judgments yield identical psychometric functions, implying that discomfort results directly from perceived asynchrony. Hypothesis 2 posited that the psychometric function is narrower for discomfort than for simultaneity, indicating that even without conscious awareness of asynchrony, subtle temporal discrepancies may still be subconsciously perceived, eliciting discomfort. Hypothesis 3 predicted the opposite—that the discomfort function is broader than the simultaneity function—implying that a certain degree of asynchrony is perceptible but not necessarily unpleasant.
The study’s results support hypothesis 3. The temporal window is wider for discomfort judgments than for simultaneity judgments, suggesting that audiovisual asynchrony can be detected without causing discomfort. This finding aligns with Fujisaki et al. (2004), who identified a perceptual category of “not simultaneous but related” using a three-alternative simultaneity task. Adaptation effects were also observed within this category. The similarity between our discomfort window and Fujisaki’s “related” window suggests that the perception of cross-modal relatedness, rather than synchrony alone, plays a key role in the emergence of audiovisual discomfort.
We proposed three hypotheses to explain the emergence of discomfort. Hypothesis 1 suggested that discomfort and simultaneity judgments yield identical psychometric functions, implying that discomfort results directly from perceived asynchrony. Hypothesis 2 posited that the psychometric function is narrower for discomfort than for simultaneity, indicating that even without conscious awareness of asynchrony, subtle temporal discrepancies may still be subconsciously perceived, eliciting discomfort. Hypothesis 3 predicted the opposite—that the discomfort function is broader than the simultaneity function—implying that a certain degree of asynchrony is perceptible but not necessarily unpleasant.
The study’s results support hypothesis 3. The temporal window is wider for discomfort judgments than for simultaneity judgments, suggesting that audiovisual asynchrony can be detected without causing discomfort. This finding aligns with Fujisaki et al. (2004), who identified a perceptual category of “not simultaneous but related” using a three-alternative simultaneity task. Adaptation effects were also observed within this category. The similarity between our discomfort window and Fujisaki’s “related” window suggests that the perception of cross-modal relatedness, rather than synchrony alone, plays a key role in the emergence of audiovisual discomfort.