Presentation Information
[O10-04]Can Earth Observation Data Become a "Natural Resource" for Musical Art? — The Resonance of String Quartet No. 1 "Polar Energy Budget"★Invited Papers
*Hiroto Nagai1 (1.Rissho University, Faculty of Geo-Environmental Science Department of Geography)
Keywords:
sonification,music,ERA5,MODIS
The conversion of Earth observation data into sound, known as audification or sonification, is opening new horizons for artistic expression in the Earth sciences. Sound unfolds along a temporal axis and engages sensory pathways distinct from visual representation, enabling intuitive perception of natural phenomena. However, many previous efforts have remained limited to mechanically mapping observed values to pitch or amplitude, with insufficient consideration of musical structure or emotional impact. Faithful mappings of geophysical variables often produce monotonous sequences or uniform outputs; although information is preserved, such representations rarely engage listeners’ cognition or emotions. This reveals a fundamental gap between physical fidelity and artistic expression.
In response to this challenge, the author composed String Quartet No. 1 “Polar Energy Budget.” In this work, time series data since 1985 were extracted from four polar locations using ERA5-Land reanalysis data (radiation and precipitation) and satellite-derived MODIS products (land surface temperature and cloud optical thickness), and converted into MIDI format. These sequences were then intentionally processed using a Digital Audio Workstation (DAW), incorporating conventional compositional techniques, and assembled into a six-minute string quartet performed and recorded by professional musicians.
The first key finding is that the “moderate irregularity” inherent in observational data imparts a unique vitality to the melody. Natural variability exists between strict periodicity and complete randomness. Subtle fluctuations in radiative fluxes generate meaningful irregularities that are difficult to create intentionally. The degree of irregularity can also be controlled through the selection of physical variables.
Second, harmonic structures such as conventional chord progressions rarely emerge naturally from observational data and must be introduced deliberately. In contrast, contrapuntal structures arise organically when melodies derived from identical variables at different geographic locations are performed simultaneously. This suggests that counterpoint provides a more natural structural framework for sonification than harmony.
Third, the impression conveyed by the music does not directly correspond to the semantic meaning of the physical variables. For example, melodies derived from temperature data do not inherently evoke sensations of warmth or cold; instead, impressions depend on compositional parameters such as scale, pitch range, tempo, and rhythm. Conventional sonification has often avoided composer intervention to preserve data purity. In this work, however, staged compositional intervention in melodic placement, temporal structure, and timbre allowed a narrative to emerge. This enables sonification to function as a storytelling medium, fostering emotional engagement and intellectual curiosity.
Following its publication in iScience (Cell Press), the work was featured in Nature, National Geographic, Forbes, and numerous newspapers, radio programs, and television broadcasts across more than 26 countries. Its reception reflects its realization as performable music using conventional instruments, and its integration of natural data and compositional design within established musical language. By incorporating the dynamics of the Earth system itself into musical structure, this work suggests a new contribution to music history.
Unlike visual graphs, sound engages listeners emotionally and directly, opening new possibilities for science outreach that fosters a sense of connection with Earth system processes. At the same time, the reproducible transformation framework developed here expands Earth observation data from scientific objects into creative resources. This presentation discusses the significance and future potential of sonification, and explores new forms of perception and expression enabled by Earth observation data.
In response to this challenge, the author composed String Quartet No. 1 “Polar Energy Budget.” In this work, time series data since 1985 were extracted from four polar locations using ERA5-Land reanalysis data (radiation and precipitation) and satellite-derived MODIS products (land surface temperature and cloud optical thickness), and converted into MIDI format. These sequences were then intentionally processed using a Digital Audio Workstation (DAW), incorporating conventional compositional techniques, and assembled into a six-minute string quartet performed and recorded by professional musicians.
The first key finding is that the “moderate irregularity” inherent in observational data imparts a unique vitality to the melody. Natural variability exists between strict periodicity and complete randomness. Subtle fluctuations in radiative fluxes generate meaningful irregularities that are difficult to create intentionally. The degree of irregularity can also be controlled through the selection of physical variables.
Second, harmonic structures such as conventional chord progressions rarely emerge naturally from observational data and must be introduced deliberately. In contrast, contrapuntal structures arise organically when melodies derived from identical variables at different geographic locations are performed simultaneously. This suggests that counterpoint provides a more natural structural framework for sonification than harmony.
Third, the impression conveyed by the music does not directly correspond to the semantic meaning of the physical variables. For example, melodies derived from temperature data do not inherently evoke sensations of warmth or cold; instead, impressions depend on compositional parameters such as scale, pitch range, tempo, and rhythm. Conventional sonification has often avoided composer intervention to preserve data purity. In this work, however, staged compositional intervention in melodic placement, temporal structure, and timbre allowed a narrative to emerge. This enables sonification to function as a storytelling medium, fostering emotional engagement and intellectual curiosity.
Following its publication in iScience (Cell Press), the work was featured in Nature, National Geographic, Forbes, and numerous newspapers, radio programs, and television broadcasts across more than 26 countries. Its reception reflects its realization as performable music using conventional instruments, and its integration of natural data and compositional design within established musical language. By incorporating the dynamics of the Earth system itself into musical structure, this work suggests a new contribution to music history.
Unlike visual graphs, sound engages listeners emotionally and directly, opening new possibilities for science outreach that fosters a sense of connection with Earth system processes. At the same time, the reproducible transformation framework developed here expands Earth observation data from scientific objects into creative resources. This presentation discusses the significance and future potential of sonification, and explores new forms of perception and expression enabled by Earth observation data.