When the original Episcopal Palace was destroyed by a fire in the 19th century, bishop Grau decided to assign the design of the new building to his friend Antoni Gaudí. The two had become friends when Grau was general vicar in the archdiocese of Tarragona and had inaugurated a church for which the architect had designed the high altar.
When Gaudí received the commission, he was still working at the Palau Güell in Barcelona, and thus he could not move to Astorga to study the terrain and the area of the new construction. He therefore asked the bishop to send him photographs to begin the new project. Gaudí sent his design, and it was approved in February 1889. The first stone was placed in June the following year.
The edifice, built in gray granite from El Bierzo, is in a neo-medieval style harmonizing with its location, including the cathedral in particular. It does, however, also feature some of the elements typical of the later Gaudí, such as the arches of the entrance with buttresses and the chimneys integrated in the side façades. Gaudí had devised a five-meter tall angel to crown the façade, but it was never mounted. The façade has four cylindrical towers and is surrounded by a ditch.
In 1893, after the death of Bishop Grau, Gaudí resigned over disagreements with the council, halting the construction for several years. The palace was completed between 1907 and 1915 by Ricardo Garcia Guereta. During the Civil War the building served as the local headquarters of the Falange. In 1956 Julià Castelltort, a Catalan, began restoration works to adapt the building as a bishop's residence. Later bishop Marcelo González Martín promoted the conversion to the current role of the palace, a museum of religious art called Museo de los Caminos, dedicated to the Way of Santiago.