McKellar was a native of Dallas County, Alabama and was graduated from the University of Alabama in 1891 and its law school in 1892. He moved to Memphis, Tennessee, and was admitted to the bar the same year. McKellar was first elected to the House in a special election in November 1911 to succeed the late George W. Gordon in the 10th Congressional District, which included Memphis. He won the seat in his own right in 1912 and was reelected in 1914 before successfully running for the Senate in 1916. He was reelected to the Senate in 1922, 1928, 1934, 1940, and 1946.

McKellar was a close ally of Memphis political boss E. H. Crump. He is the only Tennessee senator to have completed more than three full terms; except for him, Tennessee has generally not fully joined into the Southern tradition of reelecting Senators for protracted periods of service. (Before the era of popular election of U.S. Senators, Senator William B. Bate was elected to a fourth term by the Tennessee General Assembly, but died only five days into it. Senator Isham G. Harris had also died early in his fourth term. Senator Joseph Anderson was elected by the General Assembly to three full terms after completing the term of William Blount, who was expelled from the Senate.) McKellar twice served as President pro tempore of the United States Senate, commencing in 1945, being the first to hold the position under the system that has prevailed since of reserving it for the most senior member of the majority party.

In 1952 McKellar stood for a seventh term, despite being by then quite elderly (age 83). He was opposed for renomination by Middle Tennessee Congressman Albert Gore, Sr.. McKellar's reelection slogan was "Thinking Feller? Vote McKellar.", which Gore countered with "Think Some More - Vote for Gore." Gore defeated McKellar for the Democratic nomination in August in what was widely regarded as something of an upset. At this point in Tennessee history, the Democratic nomination for statewide office was still "tantamount to election", as the Republican Party's activities were still largely limited to East Tennessee, as they had been since the Civil War. Gore went on to serve three terms in the Senate.

McKellar's 1952 defeat was part of a statewide trend. 1952 also saw the defeat for renomination of incumbent governor of Tennessee Gordon Browning by Frank G. Clement. Browning, who had served a total of three terms as governor, the last two successive, had also at one point been a close ally of Crump's but had since broken ranks with him. As Clement and Gore were both considerably younger and regarded as more progressive than their predecessors, some historians cite the 1952 elections as an indication that Tennessee was earlier to enter into the "New South" era of Southern politics than most of the other Southern states. This election also marked the end of Crump having any real influence in Tennessee beyond Memphis.

McKellar wrote a book about his Tennessee predecessors in the Senate called Tennessee Senators as Seen by One of Their Successors (1942). In recent years it has been updated by one of his successors, current Senate Majority Leader Dr. Bill Frist.

Lake McKellar in the industrial area of Memphis near the Mississippi River and McKellar Airport in Jackson, Tennessee ("MKL") are both named in his honor.

Preceded by: Carter Glass (first time), Arthur Vandenberg (2nd time) President pro tempore of the United States Senate January 6, 1945-January 2, 1947; January 3, 1949-January 2, 1953 Succeeded by: Arthur Vandenberg (first time), Styles Bridges (2nd time) Preceded by: Luke Lea United States Senator From Tennessee 1917-1953 Succeeded by: Albert Gore, Sr.