Objective: Irregular patterns of marital cohabitation are a common problem in upper Egypt due to employment conditions. The objective of this study was to investigate the effect of irregular marital cohabitation on the quality of life and sexual function of infertile men. Methods: In total, 208 infertile men were included and divided into two groups. The first group included 134 infertile men with an irregular pattern of marital cohabitation and the second group included 74 infertile men with a regular pattern of marital cohabitation. All subjects were assessed through a clinical evaluation, conventional semen analysis, the fertility quality of life (FertiQoL) questionnaire, the International Index of Erectile Function (IIEF-5) score, and the premature ejaculation diagnostic tool (PMEDT). Results: The two groups were compared in terms of conventional semen parameters, FertiQoL, IIEF-5 score, and PMEDT. Infertile men with an irregular pattern of marital cohabitation had significantly lower subscale and total FertiQoL and IIEF-5 scores. Additionally, they had significantly higher PMEDT scores. Erectile dysfunction and premature ejaculation were more common in them than in infertile men with a regular pattern of marital cohabitation. Conclusion Irregular patterns of marital cohabitation had an adverse effect on quality of life and sexual function in infertile men.