Applewood Manor is one of the three oldest houses remaining from colonial Castleton. The original house was built by Noah Hoyt, younger brother of Nehemiah Hoyt (Hoit) one of the first settlers who came to Castleton as a single man in 1771. As a Green Mountain Boy, Nehemiah Hoyt assisted in the capture of Fort Ticonderoga; he was the third man to enter, following on the heels of Colonel Ethan Allen.

Noah Hoyt married Sarah Moulton in 1783 and moved to Castleton in 1784 where he was a farmer, a constable and a land speculator. He began construction of the Cape Cod home in 1789. The colonial front of the house which faces east was added in 1803. Following his death in 1811, the house passed from Noah to his son, Noah, a farmer and captain in the Vermont militia. The house remained in the Hoyt family until well into the 20th century, with the widow of John Hoyt, great grandson of the original builder, being the last member of the Hoyt family to live in the house. Since then, only two other families (Manning and Hagenlocher) have owned the house before Ralph and Nancy purchased it in 2004.

The Hoyt home has always been noted for its captivating beauty and its unique workmanship.. The character and history of the house have been lovingly preserved through careful renovations, providing an interesting and comfortable place to stay.