Born in Jefferson County, Georgia, he was raised in Athens, Georgia and attended the University of Georgia. He was admitted to the bar in 1836 and became solicitor general of the western judicial circuit of Georgia. He was elected as Democrat to the 28th, 29th, 30th and 31st Congresses.

He was chairman of the U.S. House Committee on Mileage during the 28th Congress, and Speaker of the United States House of Representatives during the 31st Congress.

He sided with President Andrew Jackson on the question of nullification; was an efficient supporter of President James K. Polk's administration during the Mexican-American War; and was an ardent advocate of slavery extension into the territories, but when the Compromise of 1850 had been agreed upon he became its staunch supporter as a Union Democrat, and on that issue was elected governor of Georgia by a large majority.

In 1851 he left the House to serve as the Governor of Georgia, holding that post until 1853. He was elected to the 34th Congress and then took the position of Secretary of the Treasury in Buchanan's Cabinet. He served for three years, resigning in December 1860. In 1860, he ceased to be a Unionist, and became a leader of the secession movement.

He was president of the convention of the seceded states which drafted a constitution for the Confederacy which assembled in Montgomery, Alabama, on February 24, 1861.

He was appointed a brigadier general in the Confederate Army February 13, 1862, and was promoted to major general September 9, 1863. His brigade played a key role in the fighting at Crampton's Gap during the Battle of South Mountain.

He surrendered at Macon, Georgia, April 20, 1864. From the close of the war until his death, he vigorously opposed the Reconstruction Acts.

He died in New York City in 1868. He is buried in Athens, Georgia.