He was born in Columbia, South Carolina in 1928, but spent most of his life in nearby Lexington County. Shortly after graduating from high school, he enlisted in the U. S. Naval Reserve, retiring as a captain in 1988. He graduated from the University of South Carolina with a degree in English in 1952, earning a law degree from the same school four years later.

Just after leaving law school, he was elected to represent Lexington County in the South Carolina House of Representatives as a Democrat. He was reelected in 1958 and 1960. However, on April 14, 1962, Spence became the first elected official at any level in South Carolina to switch to the Republican Party. He was very uncomfortable with the national Democrats' increasingly liberal platform, and also opposed a loyalty oath required by the state Democrats. On the same day, he announced that he would seek the Republican nomination for the state's 2nd Congressional District, based in Columbia. He'd been asked by several of his friends to run before his switch, but declined. He faced fellow state representative Albert Watson of Columbia in November and wasn't given much of a chance.

At the time, the Democratic Party was virtually the only party in South Carolina, and Democratic presidential candidates frequently won the state with close to 90f the popular vote. Indeed, Spence's party switch made him the first Republican in either house of the state legislature since Reconstruction. However, he only lost to Watson by five percentage points--a race that is largely credited with beginning the rise of the Republican Party in South Carolina. In 1966, Spence was elected to the South Carolina Senate, becoming the chamber's lone Republican. He was reelected in 1968, and naturally became minority leader when he was joined by other Republicans that year.

In 1970, Spence ran for the 2nd Congressional District again. Watson, who had defeated Spence eight years before, had become a Republican in 1965 and was running for governor. This time, it was Spence who won a narrow victory. He became the first freshman Republican congressman from South Carolina since Reconstruction. He was unopposed for reelection in the gigantic Republican landslide of 1972 and reelected 14 times thereafter. Early on, he established himself as one of the most conservative members of the House. His first bill in his freshman year was the first-ever balanced-budget amendment to the Constitution.

Spence had little trouble being reelected until 1986, when he faced a well-funded opponent for the first time since his first campaign in 1970. He faced another tough campaign in 1988. However, the retirement of Congressman William Whitehurst of Virginia made Spence the second-ranking Republican on the House Armed Services Committee, on which Spence had served since his first term. The district includes several military bases. He became the committee's ranking Republican in 1993 and chairman of the committee in 1995 when Republicans took control of the House. Largely because of his advances on the committee, Spence was never seriously challenged after 1988, and even ran unopposed from 1990 through 1996. As chairman, Spence emphasized military readiness, calling it "the best insurance we have both for peace and freedom." He stepped down as chairman after the 106th Congress, but remained on the committee.