He was born in Greensville County, Virginia and graduated from the University of North Carolina at Chapel Hill in 1816, and then studied law in Connecticut. In 1819 he was admitted to the Southampton County, Virginia bar.

He married the daughter of a prominent land-owner in 1821 and became a planter himself, as well as continuing as a lawyer.

He served in the Virginia House of Delegates from 1823 to 1827, was a delegate to the state constitutional convention of 1829-1830, and from 1831 to 1837 served in the United States House of Representatives (the 22nd, 23rd and 24th Congresses), chairing the U.S. House Committee on Foreign Affairs from 1835 to 1836. During this time, he was an active supporter of most elements of Andrew Jackson's presidency, but was also a staunch advocate of states' rights.

He was appointed judge of the United States District Court for the Eastern District of Virginia in 1837. He was also a delegate to the Virginia constitutional conventions of 1829 and 1850.

Mason was the U.S. Secretary of the Navy from 1844 to 1845 in President John Tyler's Cabinet and then U.S. Attorney General and then again Secretary of the Navy from 1846 to 1849, succeeding George Bancroft, under President James K. Polk.

The period of Mason's service as Navy Secretary was marked by intense Congressional pressure for economy, requiring the decommissioning of the Navy's ships of the line and making it difficult to maintain a continuous naval presence on foreign stations. The construction of floating drydocks for several Navy Yards, the simplification of the Navy's ordnance system, an expansion of the Navy's scientific endeavors and the formalization of status of the naval engineers also marked Mason's first term as Secretary.

His second term was marked by efforts to sustain the Navy's combat forces in the Gulf of Mexico and along the far-distant Pacific coast, the beginning of construction of new steamers and an effort to obtain potential warships thorough the subsidization of civilian mail steamships. The latter was an early, and ultimately unsuccessful, experiment in public-private partnership.

He was in private legal practice from 1849 to 1854 and served as President of the Virginia Constitutional Convention of 1851 and from 1853, until his death in Paris, France in 1859, the U.S. Minister Plenipotentiary to France. In this capacity he attracted attention by wearing at the court of Napoleon III a simple diplomatic uniform (for this he was rebuked by U.S. Secretary of State William L. Marcy, who had ordered American ministers to wear a plain civilian costume), and by joining with James Buchanan and Pierre Soulé, ministers to Great Britain and Spain respectively, in drawing up (Oct. 1854) the famous Ostend Manifesto.

In politics he was a typical Virginian of the old school, a states rights Democrat, upholding slavery and hating abolitionism.

After his death in Paris, his remains were conveyed to the United States and interred in Hollywood Cemetery, Richmond, Va.

USS Mason (DD-191), 1920-1940, was named in honor of Secretary of the Navy John Y. Mason.