In the Middle Ages, the manor of Hillerødsholm lay on the site of today's castle. Hillerødsholm is first mentioned in 1275 and throughout the following centuries was owned by some of the most distinguished families: Hvide, Brok Ulfeldt and Gøye. In the middle of the 1500s Birgitte Gøye and Herluf Trolle had a new main building built on one of the three islands in the lake. In 1560 Birgitte Gøye and Herluf Trolle exchanged Hillerødsholm with King Frederik II for Skovkloster near Næstved. The king's new property was one of his favourite residences and he gave it his own name - Frederiksborg.Frederik II's son, Christian IV was born at Frederiksborg Castle in 1577 and was deeply attached to it. In 1599 he instigated extensive restoration work in the course of which the old main buildings were pulled down and replaced by a magnificent new Renaissance Castle. The Castellan's House and the Chancellery were built on the outer courtyard during the years around 1613. Throughout the seventeenth century, Frederiksborg Castle was often used as a royal residence, but during the succeeding centuries the royal family seldom used it. However, the Castle was of great ceremonial importance. With the introduction of absolutism, Danish kings were no longer crowned but instead anointed in the Chapel at Frederiksborg Castle.