Abstract: Rails constitute essential components of railway tracks, with rail damage detection holding significant importance for ensuring railway safety, improving operational efficiency, and reducing maintenance costs. Addressing current rail damage detection workflow issues of low playback efficiency and missed damage reports, a YOLOv5-based object detection network incorporating variable receptive field technology was employed to expand the network’s receptive field range, emphasizing global features. Experimental results demonstrated that applying this method to actual ultrasonic testing data for intelligent rail damage analysis achieved an average damage recognition rate of 98%, with the proposed algorithm outperforming traditional methods in false positive rates, false alarm rates, and recognition efficiency.