The purpose of this study was to compare the characteristics of starters and non-starters with respect to the anthropometric and physical fitness in a Japanese university rugby team. Our sample pool consisted of 54 players of the K University rugby team, who were registered for the official games (all 7 games/year) in 2018 and 2019. The starter group (26 players) comprised of players with more than 3 games/year while the non-starter group (28 players) comprised of players that registered for official games but played less than 3 games. Anthropometric parameters, such as height, body mass, skinfold (8 items), girth (5 items), body composition, muscular strength, intermittent endurance, and sprint time, were assessed. We found that the starter group for forwards had lower skinfold (Biceps, Iliac crest, Supraspinale, Abdominal, Front thigh, and Medial calf) and percentage body fat than in the non-starter group. The starter group for backs had muscle strength of narrow chinning is higher than in the non-starter group. Finally, our study helped to clarify the physical factors that influence the difference in athletic performance between starters and non-starters in university rugby team. The forward starters had lower percentage body fat and better body composition and backs starters had better narrow chinning. The findings provide information to university rugby players who want to become starters, as well as help the coaching staff understand the factors that need to be strengthened in the players.