Appling was born in High Point, North Carolina. He attended Oglethorpe College, but left during his sophomore year when he was signed by the Southern League Atlanta Crackers in 1930. He was a good hitter in his first year, but committed 42 errors in 104 games. The Chicago Cubs showed some interest at first, but decided not to sign him, and the White Sox ended up purchasing him from the Crackers for $20,000. He seemed a poor pickup initially, as his hitting fell off and his fielding failed to improve, but in 1933 he stopped trying to be a power hitter and had his first of nine straight .300 seasons.

Appling was a good leadoff hitter who topped the .400 mark in OBP eight times (1933-1934, 1937, 1940, 1943, 1947) and drew over 100 walks three times (1935, 1939, 1949), though he often batted third due to a lack of offensive talent on the White Sox. Indeed, this lack of talent insured that Appling, who spent his entire 20-year career with the White Sox, never had a chance to play in a World Series. His best season was 1936, when he batted .388, knocked in 124 runs (his only 100-RBI season), scored 111 times, recorded 204 hits, and had a team-record 27-game hitting streak. His .388 average was good for the first batting title ever won by a shortstop and was the highest batting average recorded by a shortstop in the 20th century. Appling won another batting title in 1943 with a .328 average and also led the league in OBP that year (.419). Appling was selected to seven All-Star teams (1936, 1939-1941, 1943, 1946-47).

Appling was famous among his teammates for complaining day in and day out about minor ailments such as a sore back, a weak shoulder, or shin splints. While much of this complaining was probably for show, it earned him the moniker "Old Aches and Pains." He did suffer one serious injury: a broken leg that cost him much of the 1938 season.

Appling was well known for his ability to foul off pitches, leading to the story that he once fouled off 10 pitches in a row on purpose when ownership refused to give some baseballs to autograph because they were too expensive; he was supposedly never refused a ball again.

Appling remained a solid contributor into his forties, but ownership was dedicated to a youth movement and he retired after the 1950 season. At his retirement, Appling was the all-time leader for most games played and for double plays by a major league shortstop, and the all-time leader for putouts and assists by an American League shortstop. These records were later broken by Luis Aparicio, who also spent the majority of his career with the White Sox.

Appling was a successful minor league manager after his playing days were over, winning pennants with Memphis in the Southern Association and Indianapolis of the American Association and being named minor league manager of the year in 1952; but his only chance to manage at the major league level was as a late-season replacement for Alvin Dark as manager of the Kansas City Athletics in 1967, leaving his major league managerial record at 10-30. He later worked as a batting instructor for the Atlanta Braves in the 1980s. At age 75, he appeared in an old-timers game and hit a home run off Warren Spahn.

Appling was elected to the Baseball Hall of Fame in 1964.

He died in Cumming, Georgia at age of 83.