O'Neal, a 6 ft 11 in (2.11 m), 250 lb (113 kg) forward-center, declared his eligibility for the 1996 NBA Draft straight out of high school and was selected by the Portland Trail Blazers with the 17th pick of the first round. In Portland, O'Neal spent most of his time coming off of the bench, averaging around four points per game. He became the youngest player to play in an NBA game at the age of 18 years and one month. (That mark has since been eclipsed by Andrew Bynum of the Los Angeles Lakers.)

During the 2000 off-season, O'Neal was traded to the Indiana Pacers with Joe Kleine in exchange for Dale Davis. At Indiana, O'Neal was named a starter and became a standout player, averaging 12.9 points in his first season, three times more than any season he had at Portland. His averages continued to improve in the following seasons. In the 2001-02 season, O'Neal averaged 19.0 points, 10.5 rebounds and 2.3 blocks per game on the way to winning the NBA Most Improved Player Award. He also earned his first NBA All-Star Game selection and his first All-NBA Team selection, being selected to the All-NBA Third Team.

In the 2002-03 season, O'Neal continued to improve and became one of only three NBA players to average at least 20 points and 10 rebounds per game. He was selected to start in the NBA All-Star Game and was again selected to the All-NBA Third Team.

By the 2003-04 season, O'Neal was averaging 20.1 points and ten rebounds per game, leading the Pacers to the best record in the NBA, earning an All-NBA Second Team selection and placing third in MVP balloting. In the playoffs, he helped lead the Pacers to an Eastern Conference finals appearance against the Detroit Pistons. However, he was ineffective because of injury, and the Pacers lost to the Pistons in six games.

He was a member of the U.S.A. men's basketball team that finished a disappointing sixth place in the 2002 World Basketball Championships in Indianapolis. Though he helped the U.S. team qualify for the Olympics during the 2003 Tournament of the Americas and was a member of the 'core group' for the Olympics, a knee injury forced him to drop out of the national team that competed in the 2004 Olympics.

On November 19, 2004 however, O'Neal became involved in a massive brawl at the end of a game against Detroit, and was filmed striking a Pistons supporter on the basketball court. As a result, O'Neal was suspended indefinitely by Commissioner David Stern while the NBA investigated the incident. On November 21, O'Neal received a 25-game suspension for his part in the brawl. However, thanks to the urging of an arbitrator in just before Christmas, Jermaine won a 10-game reduction in his sentence and has since returned to action with the Pacers.

O'Neal has been called a poster child for so-called prep-to-pro players, for his patience and determination to become a star player in the NBA. He has also evolved into the team leader for the Pacers.

O'Neal is known for his outspokenness on race in the NBA. After the NBA enacted its controversial age-limit to enter the NBA Draft, O'Neal told reporters that he felt racism was the reason why the NBA has a different standard than other professional sports leagues like the NHL or Major League Baseball (where players frequently enter the draft immediately after high school.)