The fish target detection algorithm lacks a good quality data set, and the algorithm achieves real-time detection with lower power consumption on embedded devices, and it is difficult to balance the calculation speed and identification ability. To this end, this paper collected and annotated a data set named "Aquarium Fish" of 84 fishes containing 10042 images, and based on this data set, proposed a multi-scale input fast fish target detection network (BTP-yoloV3) and its optimization method. The experiment uses Depthwise convolution to redesign the backbone of the yoloV4 network, which reduces the amount of calculation by 94.1%, and the test accuracy is 92.34%. Then, the training model is enhanced with MixUp, CutMix, and mosaic to increase the test accuracy by 1.27%; Finally, use the mish, swish, and ELU activation functions to increase the test accuracy by 0.76%. As a result, the accuracy of testing the network with 2000 fish images reached 94.37%, and the computational complexity of the network BFLOPS was only 5.47. Comparing the YoloV3~4, MobileNetV2-yoloV3, and YoloV3-tiny networks of migration learning on this data set. The results show that BTP-Yolov3 has smaller model parameters, faster calculation speed, and lower energy consumption during operation while ensuring the calculation accuracy. It provides a certain reference value for the practical application of neural network.