Thomas Carnegie and his wife came to the island in the 1800s and built what is left in ruins today. The mansion was destroyed by a fire in 1959, supposedly set by a poacher, angry about being shot in the leg by a caretaker.