Following the 1808-1812 renovations, the mansion changed little during Madison’s second and final retirement from public life. Yet no retirement would ever be truly final for the man that would become known as the “Old Sage of Montpelier” and a wife that defined the role of First Lady. The most archaeologically visible testament to this entertaining is the mass quantities of ceramics, wine bottle glass, and bone recovered from Dolley's Midden just down the hill from the Temple.
During their retirement, the Madisons hosted numerous parties and guests of local, national, and international importance graced the walls and walks of Montpelier. The cost of entertaining these guests, as well as the wayward life of Dolley’s son from her first marriage, would ultimately require Dolley to sell the property in 1844 in a Sherriff’s Sale.
One notable visitor, the Marquis de Lafeyette, would not only spend time with Madison, but would also visit some of the enslaved community in the South Yard (most notably Granny Milly, who likely lived in the Stable Quarter).