The Westpoort was built around 1491 and consisted of two towers with a gate in between: the Gevangentoren on the sea side and an older gunpowder tower on the land side.
After the city expanded to the west in 1563 and the new Middelburg gate was built, the Westpoort lost its function as a city gate.
In 1610 the building was given a new function as a city prison. Until then, the prison was located in the cellars of the former town hall at Bellamy Park.
A new town hall was inaugurated in 1610 and the Prison Tower was designated as a new prison. Justice was administered in the middle building of the Westpoort
and convicted prisoners were held in the Prison Tower. Feared was 'the dark pit', a room in the basement where the most severely punished criminals ended up.