The construction of the new mansion in the Georgian style reflected the status of the Madison family in the local community and, at the time, represented one of the most elite residences in Orange County. Consisting of four rooms each on the first and second floors, and underneath the secure storage and work areas of the cellars, the house was set within a symmetric, Georgian landscape. This landscape was divided into three distinct areas:
It was originally thought that the formal yard, which was kept clear of trash, was bounded by a brick wall. This acted as both a social and physical boundary such that, immediately beyond the wall, trash was discarded outside of the view of the main house. Recent archaeological excavations, however, have proven that the formal yard was enclosed by two flanking structures. What is perhaps more interesting is that the dimensions (16x40 ft.) and orientation with respect to the mansion are consistent with other 18th-century dependent buildings such as the South Kitchen and the Stable Quarter.