Objective:To recognize virus electron microscopic (EM) images automatically with deep learning techniques and to select an appropriate network for virus EM image classification.Methods:Multiple classic convolutional neural networks, such as AlexNet, VGG, ResNet, DenseNet, SqueezeNet, MobileNet, and ShuffleNet, were used to classify the seven classes of virus electron microscope images by increasing the network depth increasing the network depth and adjusting the learning rate, adjusting the learning rate, batch size and other parameters.Results:Overall, DenseNet169 achieved the best performance with 91.9% accuracy, 90.1% sensitivity and 98.6% specificity. In particular, the model performed best in parvovirus recognition. The precision, sensitivity, specificity, and F1 of papilloma virus, herpes virus, pox virus and rotavirus, were above 90%, even close to 100%, but the precision of adenoviruses and the sensitivity of polyomaviruses were not satisfied, leading to corresponding low F1 value. At the same time, Lightweight network ShuffleNet outperformed AlexNet and VGG deep networks with fewer parameters and fewer FLOPs and could achieve comparable result to ResNet with about 15 times fewer parameters and 90 times fewer FLOPs; Compared with DenseNet, Shufflenet sacrificed recognition performance in the acceptable range to achieve about 10 times fewer parameters and 80 times fewer FLOPs.Conclusions:Deep networks DenseNet169 could realize automatic recognition of virus electron microscopic images with the optimal performance and lightweight networks shuffleNet_v2_x0_5 could achieve suboptimal performcane with fewer parameters and FLOPs. In practical applications, tradeoffs can be made between deep networks and lightweight networks based on specific conditions.