Inhofe was born in Des Moines, Iowa and moved with his family to Tulsa when he was a child. He served in the United States Army from 1955 to 1956 (though other sources claim he served from 1957 to 1958) as a private.

Several years later he became president of the Quaker Life Insurance Company. He became active in Republican politics in the mid-1960s. He was a member of the Oklahoma House of Representatives from 1967 to 1969, and a member of the Oklahoma Senate from 1969 until 1977, the last four of those years as minority leader. During his tenure there Inhofe ran twice for election to other positions: for Governor of Oklahoma in 1974, losing to Democrat David Boren, and in 1976 losing a race to represent Oklahoma's First Congressional District (which was based in Tulsa) to incumbent Democrat James R. Jones.

Inhofe's political career was revived in 1978 when he was elected mayor of Tulsa, a position he held until 1984. In 1986, he made another bid for the First District after Jones retired. This time he won, and he served there from 1987 until 1994, being handily reelected every two years in what rapidly became a strongly Republican district. He first came to national attention in 1993, when he led the effort to reform the House's "discharge provision" rule, which the House leadership had long used to bottle up bills in committee.

In 1994, Boren, who had been serving in the Senate since 1979, was elected president of the University of Oklahoma and announced he would resign as soon as a successor was elected. Inhofe won the Republican nomination for the special election that November, and swept to victory amid a strong Republican tide that saw the Republicans take both houses of Congress, as well as elect a Republican to the governorship for only the second time ever. He took office on November 17 to serve the last two years of Boren's term and won the seat in his own right in 1996. He was handily reelected in 2002.