Herman broke into the majors in 1931 with the Chicago Cubs and asserted himself as a star the following season, 1932 by having 206 hits, 102 runs and a .314 batting average. A fixture in the Chicago lineup over the next decade, Herman was a consistent hitter and solid producer. He regular hit .300 or higher (and as high as .341 in 1935) and drove in a high of 93 runs in 1936.

After a sub-standard offensive year in 1940, Herman was traded to the Brooklyn Dodgers in 1941. He had one of his finest offensive season in 1943, when he batted .330 with a .398 on base percentage and 100 runs driven in.

Herman missed the 1944 and 1945 seasons to serve in World War II, but returned to play in 1946 with the Dodgers and Boston Braves (after being traded mid-season). He was traded again prior to the 1947 season to the Pittsburgh Pirates, where he assumed managerial duties, but only played 15 games. He managed in the minor leagues, then became a major league coach with the Dodgers (1952-57), Braves (now in Milwaukee) (1958-59) and Boston Red Sox (1960-64), before managing the Red Sox to lackluster records in 1965 and 1966. He coached for the California Angels (1967) and late in his career served in player development roles with the Oakland Athletics and San Diego Padres.

Herman finished his career with a .304 batting average, 1163 runs, 47 home runs, 839 RBI and a minuscule 428 strikeouts. He won four National League pennants (in 1932, 1935, 1938 and 1941) but no World Series championships, and was 189-274 as a manager.

Herman holds the National League records for most putouts in a season by a second baseman and led the league in putouts seven times. He also shares the major league record for most hits on opening day, with five, set April 14, 1936.

Herman was inducted into the Baseball Hall of Fame in 1975.