Mangum was born in Durham County, North Carolina (then called Orange County). After graduating from the University of North Carolina at Chapel Hill in 1815, he began a law practice and entered politics. He served in the United States House of Representatives from 1823 to 1826, then, after an interlude as a superior court judge, was elected as a Democrat to the Senate in 1830.

Mangum's stay in the Democratic Party was short. He opposed President Andrew Jackson on most of the major issues of the day, including the protective tariff, nullification, and the Bank of the United States. In 1834, Mangum openly declared himself to be a "Whig", and two years later, he resigned his Senate seat.

As part of a strategy to deny the Democrats the electoral votes to win the Presidency in the 1836 election, the Whigs put forward four presidential candidates: Daniel Webster in New England, William Henry Harrison in the Border States, Hugh White in the middle South, and Mangum in the South Atlantic states. The strategy failed, as Democrat Martin Van Buren won the election, but Mangum won the state of South Carolina and its 11 electoral votes.

After a four-year absence, Mangum served two more terms in the Senate, where he was an important ally of Henry Clay. He served as president pro tempore from 1842 to 1845. In 1852, he refused an offer to be vice-president on the Whig national ticket.

Realizing that he had little chance of being re-elected, Mangum retired at the end of his second term. After the breakup of the Whigs during the mid-1850s, he joined the nativist American Party in 1856. A stroke soon afterwards ended his political career.

Mangum married Charity Alston Cain in 1819. They had five children. Their only son died in 1861 at the Battle of Manassas.