Hohenschwangau Castle was built on the remains of the fortress Schwangau, which was first mentioned in historical records dating from the 12th century.
A family of knights was responsible for the construction of the medieval fortress, and it served as the seat of the local government of Schwangau.
In 1523, the castle was described as having walls which were too thin to be useful for defensive purposes.
After the demise of the knights in the 16th century the fortress changed hands several times. The decay of the fortress continued until it finally fell
into ruins at the beginning of the 19th century.
More information at:

. http://en.wikipedia.org/wiki/Hohenschwangau_Castle