During the reign of Charles II Squire Thomas Walford purchased the Manor in 1657. Expert analysis showed that the north side, facing the Gower, was probably 16th century, but that the new owner carried out major rebuilding work around 1666. (This face of the building is no longer visible due to major alterations when the Manor was sold in 2001 for residential development.)
The house remained in the Walford family until 1800. During the 18th century they had become wealthy farmers and landowners and had the building restyled in the Queen Ann fashion, which was very much in vogue with the aristocracy. The front and sides were clad in local cut stone much as we see it today. The last Walford to live in the house was William who left Sibford in 1800. Ownership then passed through several hands until Thomas Harris sold it in 1842 for £1200. It was bought by the Society of Friends for ‘the purposes of having a school’. Then it had barns and a farmyard to the rear of the house.