Tower House is used as the Lord Mayor's office and for official functions, it was built around one of the 21 medieval bastions constructed to defend Canterbury's city wall. Although maps show a house on the site in 1832 and records suggest dwellings have been in place since the 15th century, the current House is an early Victorian building, probably dating from around 1850, but the exact date is uncertain.
The first notable owner was a Mr G Neame, who lived there from 1855 to 1873 and was followed by Mr William Howard, proprietor of Chartham Paper Mills, who resided there until 1882. Two ornate Victorian wings were added around 1870, although these were demolished when Tower House was donated to the City in 1936.