Sperm identification and selection is an essential task when processing human testicular samples for in vitro fertilization. Locating and identifying sperm cell(s) in human testicular biopsy samples is labor intensive and time consuming. We developed a new computer-aided sperm analysis (CASA) system, which utilizes deep learning for near human-level performance on testicular sperm extraction (TESE), trained on a custom dataset. The system automates the identification of sperm in testicular biopsy samples. A dataset of 702 de-identified images from testicular biopsy samples of 30 patients was collected. Each image was normalized and passed through glare filters and diffraction correction. The data were split 80%, 10%, and 10% into training, validation, and test sets, respectively. Then, a deep object detection network, composed of a feature extraction network and object detection network, was trained on this dataset. The model was benchmarked against embryologists' performance on the detection task. Our deep learning CASA system achieved a mean average precision (mAP) of 0.741, with an average recall (AR) of 0.376 on our dataset. Our proposed method can work in real time; its speed is effectively limited only by the imaging speed of the microscope. Our results indicate that deep learning-based technologies can improve the efficiency of finding sperm in testicular biopsy samples.