Varicoceles adversely impact semen quality and sperm DNA fragmentation, which typically improve with surgical repair. Some men with varicoceles have ipsilateral testicular atrophy due to damage from the varicocele. This study assessed semen quality and the sperm DNA fragmentation index (DFI) response to varicocele repair in men with ipsilateral testicular atrophy (TA) versus men with no testicular atrophy (NTA). Semen parameter values and DFI in both groups were compared preoperatively and postoperatively. The Mann-Whitney U test and the Wilcoxon signed-rank test were used where appropriate. There were 20 men in the TA group and 121 men in the NTA group with no difference in age, varicocele grade, or preoperative semen parameter values between the two groups. The NTA group had a higher preoperative DFI than the TA group. Both groups showed improvement in semen quality postoperatively, only the TA group showed a significant improvement in DFI, whereas the NTA group showed significant improvements in several parameter values and DFI. The change from preoperative to postoperative parameter values when comparing the two groups revealed a difference in total sperm motile count and DFI, with a larger mean improvement in the NTA group than in the TA group. Both TA and NTA groups showed improved semen quality and DFI after varicocele repair, but the NTA group had more improvement than the TA group. However, only total motile count (TMC) and DFI had a significantly greater mean change in preoperative to postoperative response in the NTA group than in the TA group.