An End-to-End Two-Phase Deep Learning-Based workflow to Segment Man-made Objects Around Reservoirs

Reservoirs are fundamental infrastructures for the management of water resources. Constructions around them can negatively impact their quality. Such unauthorized constructions can be monitored by land cover mapping (LCM) remote sensing (RS) images. In this paper, we develop a new approach based on DL and image processing techniques for man-made object segmentation around the reservoirs. In order to segment man-made objects around the reservoirs in an end-to-end procedure, segmenting reservoirs and identifying the region of interest (RoI) around them are essential. In the proposed two-phase workflow, the reservoir is initially segmented using a DL model. A post-processing stage is proposed to remove errors such as floating vegetation. Next, the RoI around the reservoir (RoIaR) is identified using the proposed image processing techniques. Finally, the man-made objects in the RoIaR are segmented using a DL architecture. We trained the proposed workflow using collected Google Earth (GE) images of eight reservoirs in Brazil over two different years. The U-Net-based and SegNet-based architectures are trained to segment the reservoirs. To segment man-made objects in the RoIaR, we trained and evaluated four possible architectures, U-Net, FPN, LinkNet, and PSPNet. Although the collected data has a high diversity (for example, they belong to different states, seasons, resolutions, etc.), we achieved good performances in both phases. Furthermore, applying the proposed post-processing to the output of reservoir segmentation improves the precision in all studied reservoirs except two cases. We validated the prepared workflow with a reservoir dataset outside the training reservoirs. The results show high generalization ability of the prepared workflow.

Object Detection Using Keygraphs

We propose a new framework for object detection based on a generalization of the keypoint correspondence framework. This framework is based on replacing keypoints by keygraphs, i.e. isomorph directed graphs whose vertices are keypoints, in order to explore relative and structural information. Unlike similar works in the literature, we deal directly with graphs in the entire pipeline: we search for graph correspondences instead of searching for individual point correspondences and then building graph correspondences from them afterwards. We also estimate the pose from graph correspondences instead of falling back to point correspondences through a voting table. The contributions of this paper are the proposed framework and an implementation that properly handles its inherent issues of loss of locality and combinatorial explosion, showing its viability for real-time applications. In particular, we introduce the novel concept of keytuples to solve a running time issue. The accuracy of the implementation is shown by results of over 800 experiments with a well-known database of images. The speed is illustrated by real-time tracking with two different cameras in ordinary hardware.