Born near Gallatin, Tennessee, his father, the Reverend James Gwin, was a pioneer Methodist minister, and also served as a soldier on the frontier under General Andrew Jackson.

He pursued classical studies and graduated from the medical department of Transylvania University in Lexington, Kentucky in 1828. He practiced medicine in Clinton, Mississippi until 1833 when he became the United States marshal for Mississippi, serving for one year.

He was elected as a Democrat from Mississippi during the 27th Congress of 1841 to 1843. Declining a renomination for Congress on account of financial embarrassment, he was appointed, on the accession of James K. Polk to the Presidency, to superintend the building of the new custom-house at New Orleans.

He moved to California in 1849 and participated in the California state constitutional convention the same year. He also purchased some property in Paloma, California where a gold mine was established. The Gwin Mine would eventually yield millions of dollars, providing him with a fortune to live off of.

Upon the admission of California as U.S. state, Gwin was elected as a Democrat to the United States Senate. He first served from September 9, 1850, to March 3, 1855; reelected to the United States Senate to fill the vacancy occurring at the expiration of his term, caused by the failure of the legislature to elect, and served from January 13, 1857, to March 3, 1861. During the 32nd and 33rd Congresses he was chairman of the U.S. Senate Committee on Naval Affairs. During his second term he was also a member of the U.S. Senate Committee on Finance.

While in the Senate, he secured the establishment of a mint in California, the survey of the Pacific coast, a navy yard and station, with large appropriations, and carried through the senate a bill providing for a line of steamers between San Francisco, China, and Japan, by way of the Sandwich Islands.

Gwin was outspoken in his support of slavery and was twice arrested for disloyalty during the American Civil War. He traveled to France in 1863 in an attempt to interest Napoleon III in a project to settle American slave-owners in Sonora, Mexico.

He retired to California and engaged in agricultural pursuits until his death in New York City in 1885.

He is buried in Mountain View Cemetery, Oakland, California.