“It is an error very naturally prevailing, that the retirement from public service, of which my case is an example, is a leisure for whatever pursuit might be most inviting. The truth however is, that I have rarely during the period of my public life, found my time less at my disposal than since I took my final leave of it.”
James Madison to Robert Walsh Jr., December 22, 1827.
Retiring to Montpelier after the end of his second term as President, Madison kept up what he described as “an extensive and often laborious correspondence ... which seems to be entailed on Ex-Presidents.” He also received a stream of visitors, eager to hear his first-hand accounts of the early years of the republic. Author Margaret Bayard Smith described Madison’s dinner table conversation as “a stream of history ... So rich in sentiments & facts – so enlivened by anecdotes ... He spoke of scenes, in which he himself had acted a conspicuous part & of great men, who had been actors in the same theatre.”
Madison was offered honorary offices by a number of organizations, but took an active role in only a few. He served on the Board of Visitors of Jefferson’s newly-established University of Virginia, succeeding Jefferson as rector of the University from 1826 to 1834. He served as president of the Agricultural Society of Albemarle, presenting an address which detailed his progressive farming ideas. Madison was a life member of the American Colonization Society, whose objective was to settle former slaves in the African country of Liberia. He made several donations to the American Colonization Society, including a $2,000 bequest in his will. He was unanimously elected president of the Society in 1833 and held the office until he died in 1836.
In 1829, when Madison was 78 years old and in “very feeble health,” his political friends prevailed on him to run for election as a delegate to the convention that revised the Virginia state constitution. He was the only delegate who had also participated in the 1776 convention that produced the original state constitution. Madison gave only one major speech, in which he exhorted the delegates to look for compromise rather than allow local concerns to prevent the achievement of a new constitution.
Perhaps the greatest undertaking of Madison’s retirement years was the organization of his papers, particularly his extensive notes on the secret debates of the 1787 Constitutional Convention, which were to be published after his death. A visitor to Montpelier wrote in 1839, “Mr. Madison’s works will undoubtedly be the soundest political text book of the nation, and nothing that has yet been published will have contributed more to illustrate the history of the Constitution and the characters of those who were parties for and against it.”
"I am hastening my preparations to become a fixture on my farm, where I anticipate many enjoyments; which if not fully realized, will be a welcome exchange for the fatigues and anxieties of public life."
James Madison to William Eustis, March 31, 1817
"In retirement from public life - I pray you may enjoy health, with the pleasing consolation of having faithfully discharged your duty to our beloved country."
William Prentiss to James Madison, March 7, 1817