Quillen was born in Scott County, Virginia, near the Tennessee line. He graduated from Dobyns-Bennett High School in Kingsport, Tennessee in 1934. He published a newspaper in Kingsport and became a Johnson City newspaper publisher in 1936; received a Selective Service System Class 3-A draft deferment in 1940, and he later served in the United States Navy from late 1942 to 1946, mostly aboard the Ticonderoga class aircraft carrier USS Antietam (CV-36).

Becoming a Kingsport real estate developer and bank executive following World War II, Quillen also was elected to the Tennessee House of Representatives in 1954 as a Republican, serving four terms in that body. He was a delegate to the Republican National Convention in 1956, 1964, and 1968.

In 1961, B. Carroll Reece, who had represented Tennessee's 1st Congressional District for all but six of the last 40 years, died in office. His wife, Louise, took over as a caretaker until the next election. Quillen decided not to run for a fifth term in the state house in 1962, instead seeking the Republican nomination for the Tri Cities-based 1st District. This area of the state, like most of East Tennessee, has been heavily Republican since shortly before the Civil War--in fact, Republicans had held the seat for all but four years since 1859. Under the circumstances, Quillen's election in the fall was a foregone conclusion. He was reelected 16 more times without anything resembling serious opposition. He faced no major-party opposition in 1966 and 1980, and was unopposed in 1984 and 1990. He eventually became de facto leader of the Republican Party in East Tennessee and thus a power broker in Tennessee Republican politics.

While serving as governor of Tennessee, fellow Republican Winfield Dunn incurred Quillen's wrath by opposing Quillen's long-term dream of the establishment of a medical school at East Tennessee State University. Dunn claimed that Tennessee lacked the resources to adequately staff and fund two first-rate medical schools and that more resources should instead be devoted to the existing medical school in Memphis, which was approximately 500 miles from Quillen's district. One reason for Quillen's wrath may have been that Dunn was from Memphis himself, and felt that Dunn was showing too much favoritism to his hometown. (The ETSU medical school was subsequently built anyway and is known as the James Quillen College of Medicine.) Quillen never forgave Dunn, and it came back to haunt Dunn when he ran for governor again in 1986. Quillen made it known in East Tennessee Republican circles that Dunn was not to be supported. Dunn managed to overcome Quillen's opposition and won the nomination. However, without significant support in East Tennessee, Dunn stood almost no chance against Democratic State House Speaker Ned McWherter. Only a large turnout in his former Memphis base kept the margin of defeat to under nine points.

Quillen amassed a large campaign fund due to having received many large individual and PAC contributions, but never really needed to use it given the heavy Republican tilt of the 1st and his enormous popularity. Many observers expected him to retire before it became illegal to convert such funds to personal use by merely declaring them as income and paying the income tax then due; he did not do so and continued to serve.

Quillen did decide to retire prior to the 1996 election and was succeeded by Circuit Court Judge Bill Jenkins, a fellow Republican. He had the longest unbroken tenure in the House in Tennessee history, and still only managed to introduce three legislative bills within the U.S. Congress as the original sponsor. Only Reece had been elected to more terms in the House (18 to Quillen's 17), and only Kenneth McKellar had served in both chambers longer. Quillen died on November 2, 2003 and was buried at Oak Hill Cemetery in Kingsport. His funeral was one of the largest in the state's history, attended by dignitaries from both parties across the state.