This city has a rich history.Romans were at the spot where is the castle.There was a Almoravides and Almohades period,those Arabs from the North of Africa were responsible for the military architecture.Later king D.Sancho with the Crusaders conquered the castle and the city for the first time.
In 1242 D.Paio Peres Correira took definitifely the city.
Several earthquakes destroyed the castle but since 1940 it has been reconstructed.

http://www.portugalvirtual.pt/_tourism/algarve/silves/uksilves.html