Rudding Park was originally part of the Forest of Knaresborough and still retains some of the ancient oak trees. In the early 18th century Rudding was owned in turn by Messrs Williamson of Wetherby, Craddock, James Collins (who enlarged the house and planted avenues in the park) and Thomas Wilson. In 1788 Alexander Wedderburn, Lord Loughborough, the future Lord Chancellor, acquired the estate and called in the garden designer Humphry Repton to remodel the landscape.
In 1805, the estate was purchased by the Hon. William Gordon, who demolished the original house and commissioned the building of the present house in a new location. In 1824 the estate was sold to Sir Joseph Radcliffe, Bt. with the new house still unfinished. He secured architect Robert Chantrell to oversee its completion. Once completed, the house consisted of two storeys, with no second floor or attic, and was made of ashlar with a Westmorland slate roof. London architect A.E. Purdie designed a Gothic Revival chapel which was built in 1874 for Sir Percival Radcliffe, the 3rd Baronet.
The chapel is the size of a parish church and constructed from Aberdeen granite and alabaster. Several generations of the Radcliffe family then occupied the house for 150 years.