Postdoctoral Fellow (Remote Sensing) || Deep Learning || Python || Earth Observation Expert || GeoAI || GIS || Geoinformatics
I spent two days working on and fine-tuning the #SegmentAnything #Model (SAM) for #road #extraction from #satellite #images. SAM is the first #foundational model developed by Meta #AI #Research, #FAIR. I was impressed by the results achieved by SAM with one epoch model training (precision: 0.64, recall: 0.62, f-score: 0.62, dice: 0.62). We got precision (0.95) , recall (0.53), F1-score (0.65), and dice value of 0.63 by our previously developed model (ResUNetFormer) (https://lnkd.in/g6q-VW2Z)! #deeplearning #remotesensing #roadextraction #founadionalmodels #imageprocessing #datascience #dataengineering
I saw work that SAM wasn’t great for geospatial. Either way, this is already a great challenge. Learning objective functions in segmentation that have some “per pixel” component are biased towards learning “blobby” objects with large area to perimeter ratio - and poorly on vascular threadlike features. It would be interesting to develop a différentiable cost function that focuses more on boundary correctness than bulk. We could also use the gradients of the segment masks as a mask. I feel like this must have been tried. Obviously your bigggest challenge is to improve recall. 📈
Was it full model (which encoder version?) finetune or just the classifier and the regressor? Did you try to nudge the Encoder parameters?
Have you tried extracting roads with SAMGEO, a finetuned SAM model for for remote sensing data? It might save you a lot of work. https://samgeo.gishub.org/
You may know that the extracted outcomes are best used when they are treated as a network. In this context the accuracy metric used would have less correlation with the GT. A good representation of model accuracy is defined in one of the space net challenges.
can i have the tutorial of sam finetuning
That is awesome! In one of the previous work the out-of-the-box performance for overhead imagery was very limited for SAM. Glad to see it actually performs much better upon further fine-tuning. cc Saad Lahrichi Link the post for previous work here: https://www.linkedin.com/posts/saadlahrichi_you-can-now-read-our-paper-segment-anything-activity-7191848006810271745-LNZh?utm_source=share&utm_medium=member_desktop
That is wild for one epoch. Do you have the fine tuning code public ?
please
Senior Geospatial Data Scientist / Independent Researcher
3wI shouldn't do this, but I posted the message below for awareness, so doing this here as well - please have a look at the third use case (in case you missed the call): https://www.linkedin.com/feed/update/urn:li:activity:7215256099061383168? ==> https://aiforgood.itu.int/event/launch-of-2024-itu-geoai-challenge/ Having two developed models could give you a nice advantage, and do some good at the same time (happy to chat more)