Oral C3 Al, Automation & Optimization

4:45 PM - 5:00 PM JST(7:45 AM - 8:00 AM UTC)

[GW9BIT7e-04] Automation of shadowgraph imaging analysis: detecting species and characterizing their preferred environments

*Mehul N. Sangekar (1. JAMSTEC (Japan))

Detecting marine animals in their natural environments and characterising the habitat they are found in is essential for ocean science. Especially in the age of climate change, real-time in-situ detection of new and invasive species and any information of the environment they occur in, can provide valuable insights for future predictions and mitigations. At JAMSTEC, we have developed a shadowgraphic imaging system that can observe marine animals and other marine particles in-situ with high resolution. Because the system uses parallel light for imaging, sizes of animals can be accurately estimated. This optical technique also allows us to see density mixing layers, hydrothermal plumes, particle plumes etc, which can provide useful information for characterizing marine environments. The shadowgraph camera records video at high frame rates, producing a large volume of data, which is difficult to analyze manually. Also for future use of this system for long-term in-situ autonomous monitoring, automatic semi-real-time detection of plankton and the environmental conditions is essential. For this, we have been developing an automated image analysis pipeline that uses AI and other image processing techniques to perform the task of species detection and extract environmental information from shadowgraph videos. The analysis pipeline goes through videos automatically to separate objects from the background, mainly marine particles and marine animals. Object classification is performed to identify animal species using a trained model that is further improved by adding the new detections and previously untrained labels. The particle information, combined with CTD data, is used to make particle profiles and obtain information about the environmental conditions. We have also designed an algorithm to detect patterns in shadowgraph images representing mixing layers, hydrothermal plumes, particle plumes etc using unsupervised image feature extraction and clustering. This, combined with the species detection, could provide comprehensive information about the preferred environments of marine animals.