The Palace of Gathering Excellence was built in 1420 and rebuilt in 1655. During the Ming and Qing Dynasties, it was the residence of the Empress and imperial concubines.

Cixi once lived here in 1852 when she was created 'Honourable Person'. The Empress Dowager Cixi later moved her residence to this palace in celebration of her 50th birthday.