Born near Lexington, Virginia, Lindsay attended the common schools and settled in Clinton, Kentucky in 1854. There he taught school and studied law. He was admitted to the bar and commenced practice in Clinton in 1858. During the United States Civil War, Lindsay served in the infantry in the Confederate Army from July 1861 until May 1865, after which he resumed the practice of law in Clinton.

Linsay was a member of the Kentucky Senate from 1867 to 1870. He served as judge of the Kentucky Court of Appeals from 1870 to 1878, and served as chief justice of the court from 1876 to 1878. He then resumed the practice of law in Frankfort, Kentucky. He again joined the Kentucky Senate, serving from 1889 to 1893. He then served as United States Commissioner to the World's Columbian Exposition, held at Chicago, Illinois, in 1893.

Lindsay was elected to the United States Senate to fill the vacancy caused by the resignation of John G. Carlisle. He was reelected, and served in total from February 15, 1893, until March 3, 1901, and chaired the U.S. Senate Committee on Indian Depredations and the U.S. Senate Committee on Revolutionary Claims. He was not a candidate for renomination in 1900, but instead moved to New York City and practiced law. He was appointed United States Commissioner to the Louisiana Purchase Exposition at St. Louis, Missouri, in 1901. He died in Frankfort, and was intered in the State Cemetery.