Rahim played basketball at Wheeler High School in Marietta where he was named as "Mr. Basketball" in back-to-back seasons, leading the school to a state title as a junior in 1994 (his siblings played at Wheeler too and also won several state championships). He later attended college at the University of California, Berkeley for one year, after which he was selected by the NBA's Vancouver Grizzlies with the 3rd pick in the 1996 NBA Draft.

He made an immediate impact on the Grizzlies, becoming their leading scorer, and was clearly the best player on an otherwise dismal team. In 2001, he was traded to the Atlanta Hawks where he remained the premier player for several years. He was selected to the NBA All-Star team in 2002. Abdur-Rahim was traded to the Portland Trail Blazers on February 9, 2004, along with Theo Ratliff and Dan Dickau, in exchange for Rasheed Wallace and Wesley Person. He played 1½ seasons in Portland before becoming a free agent at the end of the 2004-2005 season.

During the 2005 off-season he was traded via a sign and trade agreement (in principle) to the New Jersey Nets, but on August 4, the press conference planned to announce his arrival was postponed, and it was revealed he had failed a required physical on account of scar tissue found in his knee. The trade was put on hold, pending a second opinion from other medical sources. On August 7 Abdur-Rahim was quoted as saying "I don't feel I want to be a Net" . He felt the knee was a non issue and claimed he never missed a game in his entire career because of the knee injury. Two days later, it was announced New Jersey had decided to rescind the trade.

On August 12, Abdur-Rahim signed a free agent contract with the Sacramento Kings.

He is Muslim, and his father is an imam.