Born in Broad Run, Fauquier County, Virginia, February 2, 1883, he attended public schools and graduated from Bethel Military Academy, Warrenton, Va., in 1901. He took his LLB at the law department of the University of Virginia at Charlottesville in 1903; was admitted to the bar in 1904 and practiced in Alexandria, Va.. During World War I he was assistant general counsel to the federal Alien Property Custodian. In 1918-22 he was Commonwealth attorney of Alexandria, Virginia. He served as a judge 1922-1930, and also engaged in banking, farming, and dairying. He was elected in 1930 to Congress, and authored the anti-Communist Smith Act in 1940.

A leader of the Conservative coalition, Smith led the opposition to the National Labor Relations Board. Conservatives created a special House committee to investigate the NLRB, headed by Smith and dominated by opponents of the New Deal. The committee conducted a sensationalist investigation that undermined public support for the NLRB and, more broadly, for the New Deal. In June 1940 amendments proposed by the Smith Committee passed by a large margin in the House, due in part to the Smith's new alliance with William Green, president of the American Federation of Labor. The AFL was convinced the NLRB was controlled by leftists who supported the CIO not the AFL in organizing drives. New Dealers stopped the Smith amendments but Roosevelt gave way and replaced the CIO-oriented members on the NLRB with men acceptable to Smith and the AFL.

As chairman of the all-powerful Committee on Rules after 1955 Smith controlled the flow of legislation in the House. Speaker Sam Rayburn tried to reduce his power in 1961 with only some success, but Smith's close ties to other southern Democrats, and to Republicans, kept him in power. Smith was long the leading opponent of civil rights laws for Blacks, but he had supported equal rights for women since the 1920s, thanks to his close association with feminist leader Alice Paul. He forged an alliance with Congresswoman Martha Griffiths, a liberal feminist from Michigan, to include gender as a protected category in the Civil Rights Law of 1964. Griffith and Smith defeated the liberals of the AFL-CIO who had long opposed the Equal Rights Amendment.

He was defeated for renomination by a liberal Democrat, who in turn was defeated by a Republican who took over the seat. He resumed the practice of law in Alexandria, Va., where he died October 3, 1976; interment was in Georgetown Cemetery, Broad Run, Va.