One legend has it that a corrupt Buddhist priest from Heisen-ji (平泉寺?), a local temple, so enraged the populace that they dragged him from the temple to the sea and, at Tōjinbō, threw the priest into the sea. His ghost is still said to haunt the area.
An alternate legend says that the name Tōjinbō comes from a dissolute Buddhist monk. According to the legend, a Buddhist monk named Tōjinbō, who was disliked by everyone, fell in love with a beautiful princess named Aya. Tōjinbō was tricked by another admirer of Princess Aya and was pushed off these cliffs. The legend says that ever after that time Tōjinbō's vengeful ghost would go on a rampage around the same time every year at this place, causing strong winds and rain. Some decades later, an itinerant priest took pity on Tōjinbō and held a memorial service for him. After that, the storms ceased.