He grew up in an Italian-American neighborhood in St. Louis known as The Hill. He was a talented athlete, as was his childhood friend and competitor, Yogi Berra. When they were teenagers Garagiola was considered the better baseball prospect by pro scouts, although in his writing Garagiola is always complimentary about Berra's ability.

He was signed at age 16 by the St. Louis Cardinals organization and made his major league debut in 1946. He never quite lived up to the promise of his youth, appearing in only 676 games over 9 seasons for St. Louis, the Pittsburgh Pirates, Chicago Cubs and New York Giants. He was a mediocre hitter in the majors and featured that in his self-deprecating humor. He once told this story on himself: He knew that it was time to retire, when he was catching and his ex-teammate Stan Musial stepped into the batter's box, turned to Joe, and said, "When are you gonna quit?"

After his retirement from baseball, he wrote a book, Baseball Is a Funny Game, which sold well upon release. The book was largely a collection of humorous anecdotes surrounding his upbringing and his playing career, and showcased the folksy humorous style that became his trademark as a broadcaster. He also became a broadcaster on the flagship station of the St. Louis Cardinals, KMOX, in 1955.

As an announcer, Garagiola is best known for his almost 30 year association with NBC. He began doing national baseball broadcasts for NBC in 1961. He became a broadcaster for the New York Yankees from 1965 to 1967. He returned to broadcasting baseball for NBC from 1974 to 1988. One of his most famous calls was of the Boston Red Sox' Bernie Carbo's game-tying shot in the eighth inning of game six of the 1975 World Series' against the visiting Cincinnati Reds:

"Deep centerfield, way back, way back -- we're tied up!"

During the 1980s, Garagiola formed one of the most famous broadcasting teams of all-time when he teamed up with Vin Scully for baseball telecasts. Ironically, prior to NBC hiring Scully to be their #1 baseball play-by-play announcer, that particular role was filled by Garagiola.

Garagiola was pushed to succeed Curt Gowdy as NBC's #1 play-by-play announcer (and team with color commentator Tony Kubek) in 1976. NBC hoped that Garagiola's charm and unorthodox dwelling on the personal would stop the a decade-long ratings dive for the Game of the Week. By 1983, Garagiola stepped aside from the play-by-play duties for Vin Scully and Tony Kubek was paired with Bob Costas on NBC telecasts.

After calling the 1988 World Series with Vin Scully on NBC, Garagiola resigned from NBC Sports. NBC was on the verge of losing the television rights to cover Major League Baseball to CBS. Garagiola claimed that NBC left him "twisting" while he was trying to renegotiate his deal. Garagiola was replaced on the NBC telecasts by Tom Seaver. After leaving NBC Sports, Garagiola had a brief stint as a television commentator for the California Angels.

Besides calling baseball games for NBC, Garagiola served as the host of The Today Show from 1967 to 1973 and again from 1991 to 1992. In the late 1960s and 1970s, Garagiola also hosted the game shows Joe Garagiola's Memory Game, Sale of the Century and To Tell the Truth, as well as the Monday Night Baseball pre-game show The Baseball World of Joe Garagiola to his resume. Garagiola also gained a new form of notoriety thanks to his stints as host of the Westminster Kennel Club Dog Show. He also hosted one more game show, Strike it Rich, in 1986.

Garagiola has also become an advocate in recent years. He has been vocal, particularly to young people and athletes, in his campaign against the use of spit tobacco.

Garagiola's son, Joe Jr., went on to become the general manager for the Arizona Diamondbacks and later, senior vice president of baseball operations for Major League Baseball. Joe Sr. wasn't far behind as he became a television broadcaster for the Diamondbacks. His other son, Steve, is a broadcast journalist as well, serving as a reporter and anchor for WDIV-TV, the NBC affiliate in Detroit.