There are different theories to where Zeppo got his stage name: Groucho once said that the name was derived from the Zeppelin, a new invention at the time of his birth. However, it is more commonly suggested that the name derived from that of another vaudeville performer named Mr. Zippo, whom Herbert resembled. It is possible that both are true and that some punning was involved. (Another story tells of the time the Marxes were pretending to be gentleman farmers in order to avoid conscription into World War I. The brothers would refer to each other by "hayseed" names like Zeke and Zeb; Zeb became Zeppo.)

Zeppo appeared in the first five Marx Brothers movies, as a straight man and romantic lead, before leaving the team. According to a 1925 newspaper article, he also made a solo appearance in the Adolphe Menjou comedy A Kiss in the Dark, but no known copy of the film exists, and it is not clear if he actually appeared in the finished film. He had sufficient comic abilities to have stood in for Groucho when the brothers performed on stage, and he was reputed to be very funny offstage; but he never invented a comic persona of his own that could stand up against those of Groucho, Harpo and Chico Marx, even though the role he formerly filled would continue to exist in the brothers' remaining films.

Offstage, Zeppo had great mechanical skills and was largely responsible for keeping the Marx family car running. Zeppo later owned a company which machined parts for the war effort during World War II including the Marman clamps used to hold the Hiroshima bomb inside the Enola Gay. He also founded a large theatrical agency with his brother Gummo Marx, and invented a wristwatch that would monitor the pulse rate of cardiac patients and give off an alarm if they went into cardiac arrest.

On April 12, 1927, Zeppo married Marion Benda. The couple would adopt one child, Timothy, in 1944 and would later divorce on May 12, 1954. On September 18, 1959, Zeppo married Barbara Blakeley, whose son, Bobby Oliver, he adopted and gave his surname. Zeppo and Blakely would divorce in 1972 or 1973. Blakely would later marry singer Frank Sinatra.

The last surviving Marx Brother, Zeppo died of lung cancer in 1979 at the age of 78.