Abstract: With the popularization of prefabricated buildings, the existing damage detection methods for concrete structures are easily affected by noise environment, which is an urgent problem to be solved. Therefore, a machine vision-based damage detection method for prefabricated concrete structures was designed. The machine vision acquisition device consisted of machine vision tools including image acquisition cards and CCD cameras and a wall climbing robot. The machine vision tools were mounted on the wall climbing robot to achieve image acquisition. Firstly, the collected images were grayscale processed using the weighted average method, and the grayscale images were filtered using the threshold filtering method; Secondly, the Faster R-CNN model consisting of Fast Regional Convolutional Neural Network and Regional Recommendation Network (RPN) was used to achieve damage detection in concrete structures. The experimental results showed that this method can achieve accurate structural damage detection of stained concrete surfaces, shaded concrete surfaces, and rough concrete surfaces in noisy environments, and the detection results were robust.