Schmidt was accepted to Ohio University, (Athens, OH) in 1967 and became the school's best baseball player, mostly playing shortstop. In 1971 he was drafted in the second round by the Phillies. The draft pick immediately following him was George Brett, another great third baseman whose career would closely parallel Schmidt's.

Rapidly progressing through the minor leagues, Schmidt joined the Phillies in 1972. His first full season, 1973, Schmidt struggled and batted only .196; his season was perhaps the worst rookie season ever posted by an eventual Hall of Famer. He blossomed in 1974, however, leading the National League in home runs and demonstrating astounding prowess with the glove. A patient and powerful hitter, Schmidt was best known for hitting many home runs and drawing many bases on balls. For the rest of the 1970s Schmidt excelled at bat and with the glove, winning two more home run titles and a succession of Gold Glove Awards. He helped the usually awful Phillies win three straight division titles from 1976 to 1978, the team's first postseason appearances in 26 years.

A quiet, focused player, Schmidt demonstrated little emotion on the field and was perceived as being somewhat aloof. His relationship with Phillies fans was sometimes tumultuous early in his career, which Schmidt later regretted; it warmed late in his career. He had an unusual batting stance, turning his back somewhat to the pitcher and waving his posterior while waiting for the pitch. A strong, muscular man, Schmidt was one of the best athletes of his time; teammate Pete Rose once said, "To have his body, I'd trade him mine and my wife's and I'd throw in some cash." Unlike most power hitters, Schmidt tended to hit his homers to all fields; he is probably the most accomplished home run hitter in baseball history who had no tendency to pull the ball.

A tremendous third baseman, Schmidt had a powerful arm and was especially talented at fielding short grounders barehanded. His 404 assists in 1974 remain a record for third basemen. Schmidt also filled in at shortstop and first base when necessity demanded.

On June 10, 1974 Schmidt hit what should have been a home run, giving him 549 for his career, when he hit the public address speaker that hung 117 feet above and 329 feet away from home plate at the Astrodome in Houston. The ball hit the speakers and fell to the turf. The ground rules in the Astrodome stated that the ball was live and in play. Since Schmidt had already started his home run trot, he was held to a single. Many experts agree this ball would have traveled in excess of 500 feet.

In 1980, Schmidt elevated his game to astonishing heights, leading the league in home runs by a margin of 13 and winning the National League's Most Valuable Player Award in a unanimous vote. The Phillies reached the World Series and, for the first time in team history, won it, defeating the Kansas City Royals, led by none other than George Brett. Schmidt was selected as MVP of the World Series, hitting two homers and driving in seven runs.

In 1981, the Phillies again reached the postseason and Schmidt won his second MVP Award.

In 1983, in celebration of the team's 100th anniversary, Schmidt was voted in a fan balloting as the greatest player in the history of the franchise, even though he still had five strong seasons ahead of him, including nearly 200 home runs. That year, Schmidt led the Phillies back to the World Series, but they were defeated by the Baltimore Orioles.

In 1986, Schmidt won his third MVP Award, a record for a third baseman.

Injuries to Schmidt's knees and back caused him to miss much of the 1988 season. After a poor start to the 1989 season, Schmidt chose to suddenly announce his retirement in San Diego, on May 29. Known as "captain cool" by many in Philadelphia sports circles, Schmidt surprised many with his very emotional retirement speech during which he shed many tears. Despite his poor start and subsequent retirement, or because of it, fans again voted Schmidt to the NL All-Star team.

Over his career Schmidt set a vast array of hitting and fielding records. In addition to his MVP Awards, Schmidt won ten Gold Gloves, led the league in home runs eight times, in RBI four times, OPS five times, and walks four times. He was named to twelve All-Star teams. Schmidt finished his career with 548 home runs and 1,595 RBI, two of the many Phillies career records he holds.

In the year after his retirement, Schmidt spent one season as a member of the Phillies broadcast team on the now-defunct PRISM network. He was known as a very candid and honest broadcaster despite his limited experience in the area.

In 1995, Schmidt was inducted into the Baseball Hall of Fame with what was then the 4th-highest percentage ever, 96.52àSchmidt opted, at first, to pursue a more private lifestyle after his career, rather than becoming a manager or coach. He has written a number of articles on baseball for CBS and regularly participates in charity golf tournaments.

Schmidt has publicly expressed his thoughts on various baseball controversies. He has been a vocal advocate for the reinstatement of Pete Rose to baseball. In July 2005, he appeared on Bob Costas' HBO show Costas Now to discuss steroids, and said, "Let me go out on a limb and say that if I had played during that era I would have taken steroids... We all have these things we deal with in life, and I'm surely not going to sit here and say to you guys, 'I wouldn't have done that.'"

In 2002, Schmidt was hired by the Phillies to work for several weeks as a hitting coach, each spring training. In October 2003, Schmidt was named the manager of the Clearwater Threshers in the Florida State League, a Single A team within the Phillies minor league system. He managed them in the 2004 season and then resigned.

In 1999, he ranked number 28 on The Sporting News' list of the 100 Greatest Baseball Players, the highest-ranking third baseman, and the highest-ranking player whose career began after 1967 (Barry Bonds still being in the middle of his career). Later that year, he was elected to the Major League Baseball All-Century Team.

His uniform number 20 has been retired by the Phillies, and he has been honored with a statue outside the third base gate at the team's new home, Citizens Bank Park.