Palmer was born in Detroit, where his mother was the daughter of the third Michigan Territorial Judge James Witherell, while his father was a New England merchant who had settled in the city following the War of 1812. Palmer attended the public schools, Thompson’s Academy in Palmer (now St. Clair), and studied one year at the University of Michigan in Ann Arbor. He traveled to Spain and South America and then entered the real estate business in Detroit in 1853 and then engaged in lumbering and agricultural pursuits with his future father-in-law, Charles Merrill, beginning in 1855. He served on the first board of directors and as the first president for the Michigan Society for the Prevention of Cruelty to Animals (now known as the Michigan Humane Society).

He served on the Board of Estimates of Detroit in 1873 and was a member of the Michigan State Senate 1879-1880. He was elected as a Republican to the United States Senate and served from March 4, 1883 to March 3, 1889. He was not a candidate for reelection. He was chairman of the Committee on Fisheries in the Forty-ninth Congress, and the Committee on Agriculture and Forestry in the Fiftieth Congress. While in the Senate, he became known as an advocate for the women's suffrage movement, immigration restrictions, and homesteader rights. He is credited with coining a phrase widely adopted by latter-day reformers, Equal rights for all, special privileges to none. On February 6, 1885, he delivered a noted speech arguing in favor of an amendment to the U.S. Constitution granting women's suffrage.

Palmer was appointed United States Minister to Spain on March 12, 1889 by U.S. President Benjamin Harrison and served from June 17, 1889 to April 19, 1890. He was president of the National Commission of the World Columbian Exposition in Chicago 1890-1893. He retired to his Wayne County farm near Detroit.

Palmer and his wife, Elizabeth "Lizzie" Merrill Palmer, became known for their generous gifts to the city of Detroit. Among his activities, Palmer was one of the major benefactors of the Michigan Soldiers and Sailors Monument erected at Campus Martius. In honor of his mother, he built the Mary W. Palmer Memorial Church. He was also one of the founders and the first president of the Detroit Museum of Art (now known as the Detroit Institute of Arts), to which he contributed $16,000 and its current building stands on the site of Palmer's former home.

Lizzie Palmer also commissioned the Merrill Fountain in Campus Martius, dedicated in honor of her father. The fountain was moved to Palmer Park in 1926. She bequeathed $3 million to found the Merrill-Palmer Institute in 1916, which is a national center for child and family development and is now affiliated with Wayne State University.

In 1897, Palmer donated 140 acres (0.6 km²) of land along Woodward Avenue to the city for use as a public park. This land formed the basis of Palmer Park. Palmer had inherited the land from his grandfather James Witherell. In 1885, the Palmer's had had the prominent architecture firm of Mason & Rice design a rustic log cabin-style summer house on the land, which still remains in the park, although it is currently closed to visitors.

Palmer was a Unitarian and a member of the Freemasons. He died in Detroit and is interred in Elmwood Cemetery.