Pavement defect classification and localization using weakly supervised deep learning

Abstract

Automated detection of road defects has historically been challenging for the pavement management industry. As a result, new methods have been developed over the past few years to handle this issue. Most of these methods relied on supervised machine learning techniques, such as object detection and segmentation methods, which need a large, annotated image dataset to train their models. However, annotating pavement defects is difficult and time-consuming due to their ununiformed and complex shapes. To address this challenge, a weakly supervised pavement defect classification and localization framework using deep learning is proposed in this paper. This framework has two steps: (1) a robust hierarchical two-level classifier that classifies the defects in images, and (2) First, defects are primarily localized using a weakly supervised method. Next, the defects are segmented from the localized patches obtained in the previous step. The feature maps extracted from a weakly supervised method (i.e. Class Activation Mapping) based on the results of the first classifiers are used to train a segmentation network once (i.e. U-Net or Mask R-CNN) to localize and segment the defects in the images. Thus, the proposed framework combines the advantages of weakly supervised and supervised methods simultaneously. A dataset from Georgia State in the USA was used in the case study. The proposed framework obtained high precision of 97%, 88%, 92% and 97% for localizing the alligator, block, longitudinal and transverse cracks, respectively.