The castle was built in 1580 by the imperial and prince-bishop councilor Georg Ludwig v. Seinsheim (1514 - 1591). A representative building was created in the style and spirit of the Renaissance.
From 1865, the Seinsheimsche Schloss became the Schwarzenbergsches Amtshaus with the seat of the official administrator. After a refurbishment in 1994, there is now a cafe and a wine bar on the ground floor and the city library and the city archive on the first floor.