Barbour was born at "Frascati," near Gordonsville, Orange County, Virginia. He attended common and private schools and graduated from the College of William and Mary, Williamsburg, Virginia, in 1799. A year later he was admitted to the bar, and began practicing law in Bardstown, Kentucky. In 1801, he moved his law practice to Gordonsville in Orange County, Virginia.

Barbour started his public life as a member of the Virginia House of Delegates from 1812 to 1814. When U.S. Representative John Dawson died, Barbour won the special election to fill the seat, and served as a Republican in the U.S. House of Representatives from September 19, 1814 to March 3, 1825, reaching the office of Speaker from 1821 to 1823.

He declined to run for re-election in 1824 and turned down an offer to become the professor of law in the University of Virginia in 1825. Instead, he was appointed a judge of the general court of Virginia and served for two years, resigning in 1827 to return to his seat in the House of Representatives as a Jacksonian.

For the first two years of his second stint in the House, he was chair of the U.S. House Committee on the Judiciary. In 1829 he was president of the Virginia constitutional convention.while remaining a Representative. He turned down offers of a chancellorship and the post of U.S. Attorney General before finally resigning October 15, 1830 to accept President Jackson's appointment to be judge of the United States Circuit Court for the Eastern District of Virginia, where he served until March 17, 1836.

Barbour continued to receive offers: he refused nominations for judge of the court of appeals, for Governor, and for United States Senator. Finally, he was offered appointment to the U.S. Supreme Court. Barbour served from May 12, 1836, until his death in Washington, D.C. on February 25, 1841.

He was buried in Congressional Cemetery.