Home Artificial Intelligence Let’s attempt to locate UAV with change detection datasets and FaceNet Change detection datasets Model and concept Data augmentation The experiment Conclusions

Let’s attempt to locate UAV with change detection datasets and FaceNet Change detection datasets Model and concept Data augmentation The experiment Conclusions

7
Let’s attempt to locate UAV with change detection datasets and FaceNet
Change detection datasets
Model and concept
Data augmentation
The experiment
Conclusions

One other weekend experiment

Assuming you will have an aerial image and have no idea the situation, where it was captured, but you already know spatial resolution and direction. How could computer vision help to unravel this problem? It looks like a typical problem for fact checking, georeferencing of corrupted data or navigation.

Patch matching

The primary problem is data. There are only a couple of dataset for exactly this task.

I discovered recent research https://github.com/arplaboratory/satellite-thermal-geo-localization/tree/foremost where scientists solved similar task, but for thermal imagery.

After I looked on TorchGeo, I discovered that change detection datasets are what I would like. There are temporal image pairs, and there are diversity in landscapes and seasons.

Two temporal images

Also, it is feasible to make use of segmentation or classification datasets and aggressive augmentation.

Two augmented images

So, Kaggle — is my alternative, due I haven’t any GPU-s and it’s more convenient than Google Colab due it is feasible to run training while I’m offline.

As a model I used Mobilenet with TripletMargin loss. This idea appeared in 2015, and successful used for face verification https://arxiv.org/abs/1503.03832 by many corporations.

https://arxiv.org/abs/1503.03832

Currently, there are various other options, but old and well-known solution seems good for a primary experiments.

SIFT matching

Classic SIFT and ORB matching unfortunately lacks robustness, especially for matching images captured with different seasons and using different sensors.

I exploit the model in a sliding window fashion with stride to seek out most similar point in source image.

Sliding window concept and probability map

Also, recently there appeared loads of interesting options like https://github.com/cvg/LightGlue and there was a fairly exciting challenge on Kaggle https://www.kaggle.com/competitions/image-matching-challenge-2023 . Looks like there may very well be many great ideas for further research.

I exploit two stage data augmentation to make a model more robust:

  1. Anchor and positive samples augmented by the identical geometric transforms.
  2. Positive and negative samples augmented with same color transforms, but different color transforms from anchor image. Attempt to mimic different sensors or different seasons between anchor and positive-negative samples.

Several months ago I collected data for an additional experiment https://medium.com/p/d61b4b936d52 and it looks like I could use it.

Raw footage from UAV camera

I grabbed three frames from a footage.

I only shift-scale-rotated them and never used orthocorrection or structure from motion, assuming I even have only altitude and horizon due solution should work fast.

7 COMMENTS

  1. … [Trackback]

    […] There you will find 96352 more Info to that Topic: bardai.ai/artificial-intelligence/lets-attempt-to-locate-uav-with-change-detection-datasets-and-facenetchange-detection-datasetsmodel-and-conceptdata-augmentationthe-experimentconclusions/ […]

  2. … [Trackback]

    […] Read More on to that Topic: bardai.ai/artificial-intelligence/lets-attempt-to-locate-uav-with-change-detection-datasets-and-facenetchange-detection-datasetsmodel-and-conceptdata-augmentationthe-experimentconclusions/ […]

  3. … [Trackback]

    […] Read More on on that Topic: bardai.ai/artificial-intelligence/lets-attempt-to-locate-uav-with-change-detection-datasets-and-facenetchange-detection-datasetsmodel-and-conceptdata-augmentationthe-experimentconclusions/ […]

  4. … [Trackback]

    […] Here you will find 54954 more Info on that Topic: bardai.ai/artificial-intelligence/lets-attempt-to-locate-uav-with-change-detection-datasets-and-facenetchange-detection-datasetsmodel-and-conceptdata-augmentationthe-experimentconclusions/ […]

LEAVE A REPLY

Please enter your comment!
Please enter your name here