[4O4-IS-2b] Machine learning

2:45 PM - 3:00 PM JST(5:45 AM - 6:00 AM UTC)

[4O4-IS-2b-06] Land Use Semantic Segmentation from Multilayer Raster Image Basemap Tiles

〇Aaron Bramson^1,2 (1. GA Technologies, 2. Waseda University)

work-in-progress

Keywords:

Semantic Segmentation,Land Use,Geospatial AI,Foundation Models

Here we expand on earlier research to extract "pleasant vegetation" such as trees, bushes, gardens, and grass from readily and freely available aerial image data. Our novel contribution is the use of multiple image sources to form image layers that act as additional data bands. Systematic differences among the three overlapping images should foster improved detection by creating a richer context. Previously, using a UNet trained from scratch, these additional data layers only slightly improved results, so in this update we leverage the segmentation capabilities of a SAM2.1 backbone fine-tuned with our UNet. We find that adding SAM2.1 as a backbone only improves the segmentation results slightly (F1 up 5.7% to 75.9%) for the unlayered data, though consistently across all categories. We also find that incorporating additional layers no longer improves the results when using the SAM2.1 backbone, possibly because it requires flattening to 3 channels using a trained convolution projection layer. Adjustments to the model hyperparameters or the UNet structure that better leverage the backbone's latent image information may further improve results.