Habitation at the site of San Gimignano dates back to Etruscan times, but it wasn't until Roman times that it became fortified. (But not on a grand scale.) By 450 AD the town was under threat by followers of Attila The Hun, and was somehow saved by the bishop at the time. (How he managed to achieve this seems to have been omitted from the annals of history.) Nonetheless, as a result of that the town was renamed after him.
Unfortunately that was not the end of the troubles. Squabbles between districts and between rival clans continued to occur over the next few hundred years with many families building "tower houses" to attempt to secure the high ground in the event that things turned nasty. Eventually the town council put a limit on how high these tower houses could be but there are still a lot of them to be seen. 14 significant ones, to be precise.
Skipping ahead a bit, the town continued to prosper until 1348 when the black plague came knocking. Since it killed about half of the township (seriously, imagine everyone you know in your neighbourhood, now imagine half of them dead, tomorrow) there probably wasn't as much to fight over. San Gimignano was taken over by Firenze and became something of a backwater, but it's a really charming backwater populated by friendly people, and one that I would love the chance to get to know better one day.
But getting back to the towers, lacking their original purpose, maintenance was probably not done as assiduously as it may once have been. The result is that nature has decided to make itself at home.
I'm sure it would take little more than a good whipper-snippering, but getting up there is the problem.