Abstract: In view of the current highway pavement defect detection algorithm feature extraction is imperfect and difficult to deploy on embedded equipment, missing detection of tiny cracks and pits, this paper used the Depth Sep Conv module instead of the original C3 structure, the original CSP Darknet 53 backbone network was improved into more lightweight network structure by combining with BIFPN feature fusion ideas, the original PANet path fusion structure was improved to be a more effective weight B-PANet feature fusion structure. The experimental results showed that the B-YOLOv5 algorithm improved in this paper can not only improve the accuracy of 5.81% and double the detection speed under the same data set and experimental conditions, but also improve the missed detection problem of fine cracks and pits, and the parameter size of the model was only one eighth of YOLOv5. The B-YOLOv5 algorithm can fully meet the needs of real-time performance and be better deployed on embedded devices.