In 1618-19, Sydney, now Viscount Lisle, having disparked the Great Park, conveyed the property to Sir Thomas Smith, second son of Customer Smith.
During the 17th Century, probably after this purchase, the Palace was gradually demolished, except for parts of the north (and possible the east) ranges of the courtyard. This survived because it has a continuing use, and was eventually adapted as a farmstead.