At the foot of the hill. Shortly after passing this point I asked people about the road leading to the castle, and was informed there was not: I had to find a trail over the fields (and through bushes and such).