A native of McLeansboro, Illinois, he played college basketball at the University of Evansville, and then went on to play for his local NBA team, the Chicago Bulls, during the Bulls' formative years. He was the first player selected by the Bulls in the expansion draft, earning him the nickname "The Original Bull". Sloan was known for his tenacity on defense, and led the expansion team to the playoffs in its first season. Sloan enjoyed a respectable NBA career, playing in two All-Star Games, being named to the NBA All-Defensive First Team four times and the All-Defensive Second team twice, and helping lead the Bulls to the playoffs on various occasions and helping them win one central conference title. Sloan averaged 9.1 rebounds per game in his second season, and his career rebounding average of 7.4 rebounds per game is unusually high for a guard. Not someone to back off from contact and hustle, his career was cut short by injuries, and he turned his attention to coaching.

He coached the Bulls for a brief time, winning 94 games and losing 121.

After Frank Layden's retirement from the Utah Jazz in 1988 as head coach, the Jazz chose Sloan to be his successor. Sloan enjoyed a highly successful run of sixteen consecutive seasons of taking his team to the playoffs, and he has coached such players as Karl Malone, John Stockton, Jeff Hornacek, Antoine Carr, Tom Chambers, Mark Eaton, and Jeff Malone during the process. He joined Pat Riley and Phil Jackson as the only three coaches with ten or more seasons winning fifty or more games.

After Tom Kelly stepped down as coach of the Minnesota Twins in 2001, Sloan became the longest tenured coach in major league sports with a single franchise. He has led the team to the NBA Finals twice, losing in the 1997 and 1998 championship, both times to his old team, the Michael Jordan-led Bulls.

In February of 2003, he reached his 900th victory as an NBA coach, becoming the sixth coach in NBA history to reach that number of wins.

As of that month, he and his team were involved in a battle for the eighth spot in the NBA's western conference for that season, which would have given Sloan his seventeenth straight trip to the playoffs. Nowadays, he coaches other budding stars, such as Carlos Boozer and Andrei Kirilenko.

The Jazz were tied with the Denver Nuggets for the eighth and last spot of the playoffs with three games to go in the regular season. The Jazz lost their final two games of the season, however, making Sloan miss the playoffs for the first time in eighteen seasons as Jazz coach. His wife Bobbye died shortly after the end of the season, leading some to speculate as to the status of his career.

After leading what many expected to be a completely dismantled team to a 42-40 record in the '03-'04 NBA season, missing the playoffs by one game, he finished barely behind Hubie Brown of the Memphis Grizzlies in voting for the NBA Coach of the Year Award.