The Cave Church of Gellért Hill was founded in 1924 following a pilgrimage of Pauline monks to Lourdes. The original cave, which consisted a hollow entrance room and connecting niches, were rebuilt into a church in 1925. The church was walled up during the Communists era. Nowadays it works again as a church.

The Pest side of Budapest was named after this cave. In the Middle Ages The Slavic name of the (Gellert) Hill was Pest Hill after the large cave. The word "Pest" means cave or oven.