James was born to Charles and Elizabeth (Saunders) Flanagan in Albemarrle County near Gordonsville, Virginia. Before his tenth birthday the family moved to Boonesboro, Kentucky. As a young man he moved to Cloverport, Kentucky on the Ohio River and became a prospersous merchant. He also read law and was admitted to the Kerentuky bar in 1825. He married Polly Moorman in 1826 and the couple had several children before moving to Henderson, Texas in 1844.

Flanagan established himself in Henderson by opening a store. He bought a farm, speculated in land, and practiced law. Politically, he was a Whig and an active supporter of Sam Houston. He later became a moderate Republican. Flanagan served in the Texas House of Representatives (1851-1852) and the Texas State Senate (1855-1858).

When the Civil War came to Texas, Flanagan was a Unionist, but he withdrew to his farm and lived quietly. He returned to active politicsduring the Reconstruction. He served as a delegate to both Constitutional Conventions. The first, in 1866, produced a state constitution that was rejected by the radical Republicans in the U.S. Congress. The second, in 1868-1869, was successful.

Under the new Constitution, Flanagan was elected Lieutenant Governor. He served there only until 1870, when Texas was readmitted to the Union. The legislature named him, along with Morgan Hamilton, to the U.S. Senate. He served one term as a Senator, until 1875 when he was replaced by the Democrat Samuel Maxey. In the Senate he was a supporter of the Grant Administration.

After his Senate term, Flanagan took up residence on one of his farms near Longview, Texas. He married again (he was widowed twice and married three times), this time to Elizabeth Lane. The three marriages produced a total of eleven children, one of who was David Webster Flanagan who also became a politician in Texas.

Flanagan died on his farm in Longview in 1878 and was buried next to his first wife, Polly, in their family graveyard in Henderson, Texas.