Born in Georgia, he was known as both "Big Jawn" and "The Big Cat" for his smooth fielding around the bag at first base. He had a fine batting eye, and in his early career hit for high averages, leading the National League with a .349 batting average in 1939. In 1938 he batted .363, but Cardinals teammate Ducky Medwick took the title with a .374 average. Mize then changed targets and went for power instead of batting average. He led the National League in home runs in 1939 with 28, and in 1940 with 43, also leading the league in Runs Batted In in 1940 and 1942. At the end of the 1941 season, however, he was traded to the New York Giants by Cardinals general manager Branch Rickey, who famously believed in trading players before they reached their declining years.

Mize spent 1943, 1944 and 1945 in military service during World War II. Returning to the Giants in 1946, a broken toe caused him to fall one short of the home-run title, won by Ralph Kiner of the Pittsburgh Pirates. In 1947 he rebounded to hit 51 home runs and tie Kiner for the league lead. He also led in Runs and RBI, and became the only player to strike out fewer than fifty times while hitting fifty home runs. In 1948, Mize and Kiner again tied for the league home-run championship with 40 each. Mize was traded to the New York Yankees late in the 1949 season after expressing discontent with the amount of his playing time.

Mize spent the last 5 years of his career with the Yankees, mostly as a part-time player, ending in 1953. He was, however, considered a valuable contributor to their winning an unprecedented 5 consecutive American League pennants and World Series titles. In the 1952 World Series against the Brooklyn Dodgers, he hit 3 home runs, 1 as a pinch-hitter, and was robbed of a 4th by Dodger right fielder Carl Furillo, who made a leaping catch above the fence in the 11th inning to preserve a win for the Dodgers.

Mize holds the Major League record for the most times hitting 3 homers in one game, a feat he performed 6 times. He also was the only player to do it in both leagues - 5 times in the National League and once in the American. He finished his career with 359 home runs. Like DiMaggio, Ted Williams, and Hank Greenberg, all of whom spent at least 3 years in the military at the peak of their power, Mize undoubtedly lost a large number of home runs because of his service.

Mize had 2 notable idiosyncrasies: he was the first to smear mud under his eyes to reduce glare, and he never stepped out of the batter's box between pitches.

For a player with such notable sabermetric statistics, he was also quite late in being inducted into the Hall of Fame, finally being chosen by the Veteran's Committee 28 years after his retirement. There are at least two possible explanations for this. One, during his playing years, he apparently did not enjoy particularly good relations with the baseball sportswriters, from whose ranks are chosen those members who vote on candidates for the Hall of Fame. Two, his power, his fine batting average, and his extremely good On-Base Percentage were not as evident to his contemporaries, who were more impressed by Williams, DiMaggio, and Stan Musial, as they are today in the light of sabermetric analysis.