Abstract: Manual recognition of the type of weld defects through ultrasonic Time of Flight Diffraction (TOFD) image has disadvantages of low efficiency and reliability, due to the limitation of experience and knowledge of the tester. In order to improve the accuracy and efficiency of defect's identification, characteristics of weld defect of TOFD-D scan imagery were analyzed, and Faster Region based Convolutional Neural Networks(Faster RCNN) was proposed to auto-recognize the defect's type. Besides, the methods of proposed box optimization and sample expanding were also explored to improve training efficiency and stability. Finally, recognition effect and influences were analyzed. Research results show that D-scan image of the weld defect is closely related to the defect's geometry which can be used to distinguish the defect's type. Although the Faster RCNN network may misjudge noise to the interface, auto-recognition of the defect's type can be implemented through Faster RCNN with high recognition rate, robustness and anti-jamming ability, in which recognition accuracy of more than 97% is achieved.