The first fort in this site was built by Spanish in 1628. However, it had been destroyed by local people and Dutch in several decades after it was constructed. The existing fort was constructed by Dutchmen in 1644, called Fort Antonio. After the Second Opium Was in 1868, the fort had been occupied by British till 1972. After the refurbishment worked completed, the fort was reopened to public in 2005.