Born in Kemper County, Mississippi, Stennis received a bachelor's degree from Mississippi State University (then Mississippi A&M) in 1923, and a law degree from the University of Virginia, where he was a member of Phi Beta Kappa and Alpha Chi Rho, in 1928. While in law school, he won a seat in the Mississippi House of Representatives, where he served until 1932. Stennis served as a prosecutor from 1932-1937, and as circuit judge from 1937-1947, both for Mississippi's sixteenth judicial district.

Upon the death of Senator Theodore Bilbo in 1947, Stennis won the special election to fill the vacancy, winning the seat from a field of five candidates (including two sitting Congressmen: John E. Rankin and William M. Colmer). He was reelected in 1952, 1958, 1964, 1970, 1976, and 1982 and would remain in the Senate until 1989. From 1947 to 1978, he served alongside fellow Democrat James Eastland. They were the second-longest serving Senate duo in American history, behind only Strom Thurmond and Fritz Hollings of South Carolina.

Stennis wrote the first Senate ethics code, and was the first chairman of the Senate Ethics Committee.

Stennis' record on civil rights was mixed throughout his long career. As a prosecutor, he sought the conviction and execution of three Black men whose murder confessions had been extracted by torture. The convictions were overturned by the U.S. Supreme Court in the landmark case of Brown v. Mississippi (1936) that banned the use of evidence obtained by torture. The transcript of the trial indicates Stennis was fully aware of the methods, including flogging, used to gain confessions. In the Senate, Stennis was not as virulently racist as Eastland (and his predecessor, Bilbo). However, in the 1950s and 1960s he vigorously opposed such legislation as the Voting Rights Act, as did most of the Southern senators. He also signed the Southern Manifesto of 1954. He openly supported Barry Goldwater's presidential bid in 1964, as did most of the state's prominent Democrats.

However, by the 1980s he regularly supported legislation to extend the civil rights of women and minorities, though he opposed the Martin Luther King holiday. He also campaigned (along with Governor Bill Allain) for Mike Espy in 1986 during Espy's successful bid to become the first black congressman from the state since the end of Reconstruction. Earlier in his career, he had been the first Democrat to publicly criticize Joseph McCarthy on the Senate floor during the red scare, while Eastland supported McCarthy. On balance, he was far more supportive of civil rights than Eastland, who never really moderated his views during his career. In some ways, Stennis' record on civil rights is similar to those of Goldwater, Robert Byrd, Sam Ervin and J. William Fulbright--all of whom felt federal civil rights legislation gave the federal government too much power over the states and never personally engaged in promoting racist rhetoric. Still, Stennis shied away from supporting civil rights legislation when there was any political risk in doing so.

In 1973 Stennis was almost fatally wounded by two gunshots after being mugged outside his Washington home. In October 1973 during the Watergate scandal the Nixon administration proposed the Stennis compromise, wherein Stennis would listen to the contested Oval Office tapes and report on their contents, but this plan went nowhere.

Stennis lost his left leg to cancer in 1984.

He was unanimously selected President Pro Tempore of the Senate during the 100th Congress (1987-1989). During his Senate career he chaired, at various times, the Select Committee on Standards and Conduct, the Armed Services committee, and the Appropriations committee. It was due to his work with the Armed Services committee (1969-1980) that he became known as the "Father of America's Modern Navy."

Declining to run for re-election in 1988, Stennis retired from the Senate in 1989 at the height of his popularity and as America's longest serving Senator. He never lost an election in 60 years as an elected official. He took a teaching post at his alma mater, which he held until his death in Jackson, Mississippi at the age of 93.

At the time of Stennis' retirement, his continuous tenure of 41 years and 2 months in the Senate was second only to that of Carl Hayden. (It has since been surpassed by Strom Thurmond, Byrd, Ted Kennedy, and Daniel Inouye, leaving Stennis sixth). The John C. Stennis Space Center, the John C. Stennis National Student Congress, the aircraft carrier USS John C. Stennis, and the John C. Stennis Lock and Dam are named in his honor.

John Stennis is buried at Pinecrest Cemetery in Kemper County, Mississippi. He and his wife, the former Miss Coy Hines, had two children, John Hampton and Margaret Jane.