Born in Beaufort, South Carolina on January 12, 1944, the Philadelphia-raised Frazier won a Gold medal in boxing at the 1964 Tokyo Olympic Games. He subsequentally turned professional, defeating Woody Goss by a knockout in the first round. He won three more fights that year, all by knockout. Joe Frazier's vision in his left eye was considered legally blind. He fought entirely with his right eye throughout his career. He never revealed this fact till after his retirement. This startling revelation revealed just how great of a fighter he had been.

In 1966, Frazier was undefeated in nine fights, winning eight by knockout and a unanimous decision over Oscar Bonavena. His knockout victims included Charlie Polite and Eddie Machen.

In 1967, Frazier won all four of his fights, including a sixth round knockout of Doug Jones and a fourth round technical knockout of George Chuvalo.

In 1968, Ali had to leave his world Heavyweight title vacant, because of his protest against the Vietnam war, and the New York commission decided to hold a fight between Frazier and Buster Mathis, the winner being recognized as world champion by the state of New York. Although the fight was not widely recognized as a world championship bout by boxing fans, nevertheless, Frazier went in there and won the fight by a knockout in 11. Then, he beat Manuel Ramos of Mexico in two rounds to defend his 'title', and he also beat Bonavena in a rematch, also retaining that 'title', by a decision in 15, after being dropped twice in round one.

In what could be considered a weird twist, 1969 saw him defend his New York 'title' in Texas, beating Dave Zyglewicz by a knockout in the first, and then box an exhibition at Times Square. He finished the year knocking out Jerry Quarry in seven, once again defending the New York 'title'.

On February 16, 1970, Frazier finally became a recognized champion when WBA world Heavyweight champion Jimmy Ellis defended against Frazier at the Madison Square Garden and Frazier defeated him with a fifth round knockout. Frazier was immediately recognized as the undisputed world champion. In his first defense, he went to Detroit to fight legendary world Light Heavyweight champion Bob Foster, who set a record for the number of defenses in the Light Heavyweight division. Frazier retained the title by a knockout in two, and then came what was nicknamed the Fight Of The Century, his first fight with Ali.

On March 8 of 1971, also at the Madison Square Garden, Frazier and Ali boxed the first of three epic bouts. In front of a world wide television audience, and an in-house audience which included such luminaries as Frank Sinatra (acting as a photographer for Newsweek magazine) and Woody Allen, Frazier dropped Ali in round 15, to secure a 15 round decision win and retain the title, inflicting Ali's first professional defeat. Around this time, many people in the Black community felt Frazier was not a vocal advocate of the Black community's struggle for civil rights and he began to be labeled an Uncle Tom by some of the members of that community.

Immediately after the first Ali fight, Frazier faced health problems, and on March 16, he was hospitalized in Philadelphia with high blood pressure and kidney problems. This caused the media to speculate that Frazier had died, and the rumor spread quickly across the United States. Frazier was released from the hospital about a week later.

In 1972, Frazier retained the title twice, beating Terry Daniels and Ron Stander, both four round knockout losers to Frazier.

Then came January 22 of 1973 in Kingston, Jamaica, and it was Frazier's turn to lose his undefeated record, and world championship belt, against George Foreman, when Foreman knocked him out in two rounds. This was the first fight ever telecast on HBO Boxing. Frazier then returned to his winning ways with a 12 rounds decision win over Joe Bugner, a future challenger to Ali for the world title. The fight was held in London.

In 1974, his second fight against Ali took place, once again in New York, where Ali reversed the result of the first fight with a 12 round decision win over Frazier. Frazier finished that year with another rematch, knocking Quarry out in five rounds.

1975 was once again, a year of rematches for Frazier, but this time around, they were accompanied by more traveling to far away countries. He met former world champion Ellis in Melbourne, Australia and came out the winner by a knockout in nine. That win made him again the number one challenger for the world crown, taken over by Ali after beating Foreman by knockout in eight at The Rumble in the Jungle. Ali and Frazier met for the third time in Quezon City, the Philippines, and Ali took every opportunity to mock Frazier, nicknaming him The Gorilla and trying to aggravate him at every chance he had. The fight, which was attended by Philippine President Ferdinand Marcos, caused a widespread media frenzy and was seen live in many countries around the world. Ali retained his title when Frazier failed to answer the bell for the final round as his right eye had swollen up and he was practically fighting blind. Ali described the fight as "the closest thing to death" he knew of.

In 1976, Frazier lost to Foreman in a rematch and retired. He made a cameo appearance in the movie Rocky and he dedicated himself to training many local boxers in his native Philadelphia, including many of his own children.

In 1980, Frazier was honoured by the band Bruford on their LP Gradually Going Tornado.

In 1981, Frazier attempted a comeback which lasted only one fight, drawing in ten rounds with Jumbo Cummings in Chicago, Illinois.

Ever since, Frazier, who likes to meet fans and sign autographs, has involved himself in different endeavours. Among the many of his sons that turned to boxing as a career, he helped train Marvis Frazier, a challenger for Larry Holmes's world Heavyweight title, and currently trains his daughter, Jackie Frazier-Lyde.

In 1990, Frazier joined Ali, Foreman, Norton and Holmes to appear on the cover of a boxing game, Champions Forever, produced for the Sega and Nintendo game systems. He appeared as himself in an episode of The Simpsons in 1992 in which he would have gotten beaten up by Barney Gumble, but objected so that he would beat up Gumble instead. He later appeared in another episode of the Simpsons in early 2006.

Frazier wrote an autobiography entitled Smokin' Joe. He was widely criticized for the many vituperative things he said about Ali in his book. He went on and on about Ali's "insults" and said he had no pity for him in his current poor health. He said he would love to open the graveyard and bury Ali when he died. In contrast, Ali's autobiography (most likely ghost-written) was very generous towards Frazier.

Frazier had a record of 32 wins, 4 losses and 1 draw, with 27 knockout wins. He is a member of the International Boxing Hall Of Fame.

Frazier is still training fighters, and enjoying his earnings from his days as a boxer at his Philadelphia mansion. Reportedly, he and Ali have recently attempted a reconciliation.

Frazier had a lawsuit overthrown by a judge, on November 18 of 2003, concerning 140.397 acres (568,000 m²) of then-farmland that he had acquired for 843,000 dollars in 1973 and had agreed in 1978 to sell for 1.87 million dollars. The current value of the land is substantially greater than in 1978 now that the land has been parceled and 476 residences have been built on it.

Although Frazier had received annual payments for the property, he says that his signature was forged on the documents and he had no knowledge of the sale. Frazier said he was due 1.3 million dollars in payments that were never made to him by the realty trust, and was asking for return of the land to his possession, plus the current value of the land (84 million dollars) in damages. Frazier discovered that the mortgage was not secured only after the trust defaulted.

An earlier claim against the current residents of the land was also thrown out, for being directed against innocent parties. His daughter Jackie, who is also a lawyer, represented him during the lawsuit.