The building started out as a palace but in 1555 it was sold to the sister of Felipe II who founded the convent four years later.