Born in Dolphsburg, New York on October 19, 1835, Dolph attended Genesee Wesleyan Seminary, studied law, and was admitted to the bar in 1861. He set up practice in Schuyler County, New York. In 1862 he enlisted in the Oregon Escort which protected settlers traveling to Oregon from Native Americans.

He settled in Portland, Oregon, becoming the cities attorney from 1864 to 1865 and a United States district attorney from 1865 to 1868. He was elected to the Oregon State Senate in 1866, 1868, 1872, and 1874. In 1882 he was elected to the United State Senate and reelected in 1888. He became the chairman of the United States Senate Committee on Coast Defenses in the Forty-ninth through Fifty-second Congresses and the United States Senate Committee on Public Lands in the Fifty-second Congress. In total he served from March 4, 1883 to March 3, 1895, being an unsuccessful candidate for reelection in 1894. He moved back to Portland where he died on March 10, 1897. Dolph was interned there in Riverview Cemetery.