Damage Vision Mining Opportunity for Imbalanced Anomaly Detection

In past decade, previous balanced datasets have been used to advance algorithms for classification, object detection, semantic segmentation, and anomaly detection in industrial applications. Specifically, for condition-based maintenance, automating visual inspection is crucial to ensure high quality. Deterioration prognostic attempts to optimize the fine decision process for predictive maintenance and proactive repair. In civil infrastructure and living environment, damage data mining cannot avoid the imbalanced data issue because of rare unseen events and high quality status by improved operations. For visual inspection, deteriorated class acquired from the surface of concrete and steel components are occasionally imbalanced. From numerous related surveys, we summarize that imbalanced data problems can be categorized into four types; 1) missing range of target and label valuables, 2) majority-minority class imbalance, 3) foreground-background of spatial imbalance, 4) long-tailed class of pixel-wise imbalance. Since 2015, there has been many imbalanced studies using deep learning approaches that includes regression, image classification, object detection, semantic segmentation. However, anomaly detection for imbalanced data is not yet well known. In the study, we highlight one-class anomaly detection application whether anomalous class or not, and demonstrate clear examples on imbalanced vision datasets: blood smear, lung infection, hazardous driving, wooden, concrete deterioration, river sludge, and disaster damage. Illustrated in Fig.1, we provide key results on damage vision mining advantage, hypothesizing that the more effective range of positive ratio, the higher accuracy gain of anomaly detection application. In our imbalanced studies, compared with the balanced case of positive ratio 1/1, we find that there is applicable positive ratio, where the accuracy are consistently high.

Disaster Anomaly Detector via Deeper FCDDs for Explainable Initial Responses

Extreme natural disasters can have devastating effects on both urban and rural areas. In any disaster event, an initial response is the key to rescue within 72 hours and prompt recovery. During the initial stage of disaster response, it is important to quickly assess the damage over a wide area and identify priority areas. Among machine learning algorithms, deep anomaly detection is effective in detecting devastation features that are different from everyday features. In addition, explainable computer vision applications should justify the initial responses. In this paper, we propose an anomaly detection application utilizing deeper fully convolutional data descriptions (FCDDs), that enables the localization of devastation features and visualization of damage-marked heatmaps. More specifically, we show numerous training and test results for a dataset AIDER with the four disaster categories: collapsed buildings, traffic incidents, fires, and flooded areas. We also implement ablation studies of anomalous class imbalance and the data scale competing against the normal class. Our experiments provide results of high accuracies over 95% for F1. Furthermore, we found that the deeper FCDD with a VGG16 backbone consistently outperformed other baselines CNN27, ResNet101, and Inceptionv3. This study presents a new solution that offers a disaster anomaly detection application for initial responses with higher accuracy and devastation explainability, providing a novel contribution to the prompt disaster recovery problem in the research area of anomaly scene understanding. Finally, we discuss future works to improve more robust, explainable applications for effective initial responses.

Wooden Sleeper Deterioration Detection for Rural Railway Prognostics Using Unsupervised Deeper FCDDs

Maintaining high standards for user safety during daily railway operations is crucial for railway managers. To aid in this endeavor, top- or side-view cameras and GPS positioning systems have facilitated progress toward automating periodic inspections of defective features and assessing the deteriorating status of railway components. However, collecting data on deteriorated status can be time-consuming and requires repeated data acquisition because of the extreme temporal occurrence imbalance. In supervised learning, thousands of paired data sets containing defective raw images and annotated labels are required. However, the one-class classification approach offers the advantage of requiring fewer images to optimize parameters for training normal and anomalous features. The deeper fully-convolutional data descriptions (FCDDs) were applicable to several damage data sets of concrete/steel components in structures, and fallen tree, and wooden building collapse in disasters. However, it is not yet known to feasible to railway components. In this study, we devised a prognostic discriminator pipeline to automate one-class damage classification using the deeper FCDDs for defective railway components. We also performed sensitivity analysis of the deeper backbone and receptive field based on convolutional neural networks (CNNs). Furthermore, we visualized defective railway features by using transposed Gaussian upsampling. We demonstrated our application to railway inspection using a video acquisition dataset of railway track in forward view that contains wooden sleeper deterioration in rural railways. Finally, we examined the usability of our approach for prognostic monitoring and future work on railway component inspection.

One-class Damage Detector Prototyping Fully-Convolutional Data Description for Prognostics

It is important for infrastructure managers to maintain a high standard to ensure user satisfaction during a lifecycle of infrastructures. Surveillance cameras and visual inspections have enabled progress toward automating the detection of anomalous features and assessing the occurrence of the deterioration. Frequently, collecting damage data constraints time consuming and repeated inspections. One-class damage detection approach has a merit that only the normal images enables us to optimize the parameters. Simultaneously, the visual explanation using the heat map enable us to understand the localized anomalous feature. We propose a prototype to automate one-class damage detection using the fully-convolutional data description (FCDD). We also visualize the explanation of the damage feature using the up-sampling-based activation map with the Gaussian up-sampling from the receptive field of the fully convolutional network (FCN). We demonstrate it in experimental studies: concrete damage and steel corrosion and mention its usefulness and future works.

One-class Damage Detector Using Fully-Convolutional Data Description for Prognostics

It is important for infrastructure managers to maintain a high standard to ensure user satisfaction during a lifecycle of infrastructures. Surveillance cameras and visual inspections have enabled progress toward automating the detection of anomalous features and assessing the occurrence of the deterioration. Frequently, collecting damage data constraints time consuming and repeated inspections. One-class damage detection approach has a merit that only the normal images enables us to optimize the parameters. Simultaneously, the visual explanation using the heat map enable us to understand the localized anomalous feature. We propose a civil-purpose application to automate one-class damage detection using the fully-convolutional data description (FCDD). We also visualize the explanation of the damage feature using the up-sampling-based activation map with the Gaussian up-sampling from the receptive field of the fully convolutional network (FCN). We demonstrate it in experimental studies: concrete damage and steel corrosion and mention its usefulness and future works.

MN-Pair Contrastive Damage Representation and Clustering for Prognostic Explanation

It is essential for infrastructure managers to maintain a high standard to ensure user satisfaction during daily operations. Surveillance cameras and drone inspections have enabled progress toward automating the inspection of damaged features and assessing the health condition of the deterioration. When we prepare a pair of raw images and damage class labels, we can train supervised learning toward the predefined damage grade, displacement. However, such a damage representation does not constantly match the predefined classes of damage grade, hence, there may be some detailed clusters from the unseen damage space or more complex clusters from overlapped space between two damage grades. The damage representation has fundamentally complex features, consequently, all the damage classes could not be perfectly predefined. Our proposed MN-pair contrastive learning method enables us to explore the embedding damage representation beyond the predefined classes including more detailed clusters. It maximizes the similarity of M-1 positive images close to the anchor, and simultaneously maximize the dissimilarity of N-1 negative ones, using both weighting loss functions. It has been learning faster than the N-pair algorithm, instead of using one positive image. We propose a pipeline to learn damage representation and use density-based clustering on the 2-D reduction space to automate finer cluster discrimination. We also visualize the explanation of the damage feature using Grad-CAM for MN-pair damage metric learning. We demonstrate our method in three experimental studies: steel product defect, concrete crack of deck and pavement, and sewer pipe defect and mention its effectiveness and discuss potential future works.

Water Surface Patch Classification Using Mixture Augmentation for River Scum Index

Urban rivers provide a water environment that influences residential living. River surface monitoring has become crucial for making decisions about where to prioritize cleaning and when to automatically start the cleaning treatment. We focus on the organic mud, or "scum" that accumulates on the river's surface and gives it its peculiar odor and external economic effects on the landscape. Because of its feature of a sparsely distributed and unstable pattern of organic shape, automating the monitoring has proved difficult. We propose a patch classification pipeline to detect scum features on the river surface using mixture image augmentation to increase the diversity between the scum floating on the river and the entangled background on the river surface reflected by nearby structures like buildings, bridges, poles, and barriers. Furthermore, we propose a scum index covered on rivers to help monitor worse grade online, collecting floating scum and deciding on chemical treatment policies. Finally, we show how to use our method on a time series dataset with frames every ten minutes recording river scum events over several days. We discuss the value of our pipeline and its experimental findings.

VAE-iForest: Auto-encoding Reconstruction and Isolation-based Anomalies Detecting Fallen Objects on Road Surface

In road monitoring, it is an important issue to detect changes in the road surface at an early stage to prevent damage to third parties. The target of the falling object may be a fallen tree due to the external force of a flood or an earthquake, and falling rocks from a slope. Generative deep learning is possible to flexibly detect anomalies of the falling objects on the road surface. We prototype a method that combines auto-encoding reconstruction and isolation-based anomaly detector in application for road surface monitoring. Actually, we apply our method to a set of test images that fallen objects is located on the raw inputs added with fallen stone and plywood, and that snow is covered on the winter road. Finally we mention the future works for practical purpose application.