Withers began her career as a child actress, first on local radio broadcasts in Atlanta, Georgia as "Dixie's Dainty Dewdrop". By the age of four, she was singing and imitating adult celebrities. In the early 1930s Withers and her family moved to Hollywood; she worked as an extra and a bit part player in several films in 1932 and 1933.

Withers' big break came when she landed a supporting role in the 1934 Shirley Temple film Bright Eyes. Her character, Joy Smythe, was mean and obnoxious, a perfect foil to Temple's sweet personality. She received positive notices for her work, and was awarded a long-term contract with Fox. Through the remainder of the 1930s she appeared in several movies every year, including Ginger (1935), The Farmer Takes a Wife (1935) and Little Miss Nobody (1936). Withers often played characters that were similar to Joy Smythe: bratty and loud.

Withers's career slowed in the 1940s, although she appeared in 16 films over the course of that decade. In 1947, recently married and in her early twenties, Withers retired for several years from acting. In 1956, she had a supporting role in the film Giant.

In the 1960s, Withers gained fame again as "Josephine the Plumber," a character in a long-running and popular series of television commercials for Comet cleansing powder that lasted into the 1970s. She has continued to do voice-over work and occasionally guest stars on television shows.