Born in Baldwin County, Georgia, Vinson graduated from Georgia Military College and studied law at Mercer University Law School starting in 1900. In 1909, at the age of 25, Vinson was elected to the Georgia House of Representatives and served as Speaker Pro Tempore by 1911. After losing a third term following redistricting, he was briefly appointed judge of the Baldwin County court, but following the sudden death of Senator Augustus Bacon, Representative Thomas W. Hardwick was nominated to fill Bacon's Senate seat and Vinson announced his candidacy for Hardwick's U.S. House seat, which he won over three other opponents. He was sworn in as the youngest member of Congress on November 3, 1914.

During Vinson's tenure in the U.S. House, Vinson was a champion for national defense and especially the U.S. Navy. By 1931, Vinson had become chairman of the House Naval Affairs Committee. In 1934, Vinson helped push the Vinson-Trammell Act, along with Sen. Park Trammell of Florida. The bill, which circumvented the regulations set up by the London Naval Conference, helped to replace the U.S. Navy's aging fleet with new warships that helped to generate shipyard jobs at the height of the Great Depression. The ambitious program also helped the U.S. Navy as the country entered World War II in 1941, as new ships were able to immediately match the latest ships from Japan.

A strong supporter of William F. Halsey, Vinson on several ocassions blocked the final appointment to Fleet Admiral, Admiral Raymond A. Spruance. By act of Congress, Admiral Spruance was granted full pay from retirement until his death.

Following the war, the House Naval Affairs Committee was merged with the Military Affairs Committee to become the House Armed Services Committee and Vinson served as the new committee's first chair, a position he would hold until his retirement in 1965. As chairman, Vinson oversaw the modernization of the military as the military's focus shifted to the Cold War. He oversaw the procurement of the first nuclear-powered aircraft carriers starting with the USS Enterprise in the late 1950s.

For his efforts, a nuclear-powered aircraft carrier was named for him, the USS Carl Vinson and Vinson became the first living American to have a U.S. Navy ship named for him. On March 15, 1980, at age 96, he attended the ship's dedication ceremonies. In addition, Vinson Massif, Antarctica's highest mountain is also named after him.

On June 1, 1981, Congressman Carl Vinson died at the age of 97.

Carl Vinson served 26 consecutive terms in the U.S. House, rarely running against significant opposition. He served for 50 years and one month, a record for the U.S. House that stood until 1994, when the mark was surpassed by Rep. James Whitten of Mississippi. In 1964, President Lyndon Johnson awarded Vinson with the Presidential Medal of Freedom with Special Distinction, the highest award the President can give to a civilian.

While Vinson never had any children of his own, his grand-nephew, Sam Nunn, served as a Senator from Georgia for 25 years. Nunn followed in his grand-uncle's footsteps, serving on the Senate Armed Services Committee for nearly his entire tenure in the Senate.