Quincy was born in Boston on February 4, 1772. He studied at Phillips Academy, Andover, graduated at Harvard in 1790, studied law, and was admitted to the bar in 1793, but was never a prominent advocate. He became a leader of the Federalist party in Massachusetts; was an unsuccessful candidate for the United States House of Representatives in 1800; served in the Massachusetts Senate in 1804-5.

From 1805-1813 he was a member of the United States House of Representatives where he was one of the small, Federalist minority. He attempted to secure the exemption of fishing vessels from the Embargo Act, urged the strengthening of the United States Navy, and vigorously opposed the erection of Louisiana into a state in 1811. In this last matter he stated as his deliberate opinion, that if this bill passes, the bonds of this Union are virtually dissolved; that the States that compose it are free from their moral obligations to maintain it; and that, as it will be the right of all, so it will be the duty of some to prepare definitely for a separation, amicably if they can, violently if they must. This was probably the first assertion of the right of secession on the floor of Congress.

Quincy left Congress because he saw that the Federalist opposition was useless, and thereafter was a member of the Massachusetts Senate until 1820; in 1821-22 he was a member and speaker of the state House of Representatives, from which he resigned to become judge of the municipal court of Boston.

From 1823-1828 he was mayor of Boston. In his term Quincy Market was built, the fire and police departments were reorganized, and the city's care of the poor was systematized.

From 1829-1845 he was President of Harvard University, of which he had been an overseer since 1810, when the board was reorganized. He has been called "the great organizer of the university." He gave an elective (or "voluntary ") system an elaborate trial; introduced a system of marking (on the scale of 8) on which college rank and honors, formerly rather carelessly assigned, were based; first used courts of law to punish students who destroyed or injured college property; and helped to reform the finances of the university. During his term Dane Hall (for law) was dedicated, Gore Hall was built, and the Astronomical Observatory was equipped.

His last years were spent principally on his farm in Quincy, Massachusetts, where he died on July 1, 1864.