Born in Durham, a town in Greene County, New York, he was admitted to the bar in 1840 and practiced in Durham, where he was elected to his first political office as town supervisor in 1842. He was appointed the Greene County District Attorney in 1844, and then elected surrogate and county judge in 1846, but then lost reelection in 1851.

He moved to Albany in 1853 and entered into partnership with former United States Representative Rufus Wheeler Peckham in 1855. Three years later, he was elected the Attorney General of New York, and then lost reelection. He also ran unsuccessfully as the Republican candidate for Lieutenant Governor of New York in 1862. He was elected to the New York State Assembly in 1866, and served as Speaker in 1867. In 1868, the year his term in office ended, he served as a delegate to the Republican National Convention.

In 1872, Tremain was elected as a Republican to the Forty-third Congress, defeating the incumbent Samuel Sullivan Cox. He served from March 4, 1873 through March 3, 1875, and then did not seek reelection. In 1873, Tremain also served with his partner's oldest son, Wheeler Hazard Peckham, as special counsel to the State in the prosecution of Boss Tweed. After leaving Congress, Tremain returned to private legal practice in Albany and then died in New York City while visiting. He was bured in Albany Rural Cemetery in Menands, just north of Albany.

Tremain's son, Lyman (June, 1843 - February 6, 1865) was a lieutenant colonel of the 10th New York Cavalry during the Civil War, who was killed at the Battle of Hatcher's Run.