Todo Yoshimasu was one of the Kampo doctors of the Koho school in the Edo period, and created the basic structure of the modern traditional Japanese Kampo medicine. His achievements are still recognized today. His oldest son, Nangai Yoshimasu, is also credited with inheriting and developing the medical techniques established by his father. Todo’s descendants continued to be involved in the medical profession. In this paper, we first report that the building of the Yoshimasu clinic still exists in Tsuwara, Kurayoshi City, Tottori Prefecture, where Shiho Yoshimasu, the fifth-generation descendant of Todo, and his son Ryozo practiced medicine. We also found the tombstones there for Tetsutaro (Shiho), Yutaro, Ryozo, Tamenori, and Teruo of the Yoshimasu family. In conclusion, one of the origins of the Yoshimasu’s, a well-known family of doctors in Japan, is in Tottori. Therefore, Tsuwara, Kurayoshi City, Tottori Prefecture is an important place to describe Todo Yoshimasu. Here, we revealed the Yoshimasu family in Tottori based on new materials.