As an outfielder and left-handed batter, Herzog played for the Washington Senators (1956-58), Kansas City Athletics (1958-60), Baltimore Orioles (1961-62) and Detroit Tigers (1963). In eight seasons, Herzog batted .254 with 25 homeruns, 172 RBI, 213 runs, 60 doubles, 20 triples, and 13 stolen bases in 634 games. In reference to his success as a player versus his success as a manager, Herzog once said, "Baseball has been good to me since I quit trying to play it."

Herzog started his managerial career with the Texas Rangers (1973), following with California Angels (1974), Kansas City Royals (1975-79) and St. Louis Cardinals (1980-90). He had his greatest success in Kansas City, where he won three straight division titles from 1976 to 1978, and in St. Louis, where he won the 1982 World Series and the National League Pennant in 1985 and 1987. In total, he led six division winners, three pennant winners, and one World Series winner in compiling a 1281-1125 career record.

Herzog's style of play, which was nicknamed "Whiteyball," concentrated on pitching, speed, and defense to win games rather than home runs. Herzog's lineups generally consisted of one or more base-stealing threats at the top of the lineup, with a power threat such as George Brett or Jack Clark hitting third or fourth, protected by one or two hitters with lesser power, followed by more base stealers. This tactic kept payrolls low while allowing him to win a lot of games in stadiums with deep fences and artificial turf, both of which were characteristics of Royals Stadium (now Kauffman Stadium) and Busch Stadium during his managerial career.

A less noticed (at the time) aspect of Herzog's offensive philosophy was his preference for patient hitters with high on base percentages: such players included Royals Brett, Hal McRae, and Amos Otis, and Cardinals Clark, Keith Hernandez, and Ozzie Smith, as well as Darrell Porter, who played for Herzog in both Kansas City and St. Louis.