Albertson was born in Malden, Massachusetts to Jewish parents - Flora Craft, a Russian immigrant, and Leo Albertson, a Polish immigrant. He had an older sister, Mabel Albertson, and a brother, Frank, who were also actors. Albertson dropped out of high school and traveled to New York City in an attempt make it big in show business. He was too poor to get a room in a flophouse, so in the winter he would sleep on the IRT subway for a nickel, and hide out when the transit workers would clear out the train at the end of the line. In the summer he would sleep in Central Park. His first real job in show business was with a vaudeville road troupe, the Dancing Verselle Sisters.

Albertson soon worked in burlesque as a hoofer (soft shoe dancer) and straight man to Phil Silvers on the Minsky's Burlesque Circuit. Besides vaudeville and burlesque, he appeared on the stage in many Broadway plays including High Button Shoes, Top Banana, The Cradle Will Rock, Make Mine Manhattan, Show Boat, Boy Meets Girl, Girl Crazy, Meet the People, The Sunshine Boys (for which he received a Tony Award nomination for Best Actor), and The Subject was Roses (for which he won a Tony for Best Supporting Actor). He was also known for two radio programs, Just Plain Bill and The Jack Albertson Comedy Show.

Jack Albertson had a prolific movie career as well, mostly playing supporting roles. Notable among these were:

a bit part as the postal worker who redirected all of Santa Claus' mail to the New York Courthouse in Miracle on 34th Street (1947) a film version of Top Banana (1953), teamed again with his friend Phil Silvers The Subject was Roses (1968), for which he won a best supporting actor Oscar Charlie's Grampa Joe in Willy Wonka and the Chocolate Factory (1971) The Poseidon Adventure (1972, where he was the husband who encouraged Shelley Winters to swim for safety in her Oscar-nominated role) Other films include:

Big Business Girl (1931) Don't Go Near the Water (1957) Monkey on My Back (1957) Teacher's Pet (1958) Period of Adjustment (1962) Days of Wine and Roses (1962) Kissin' Cousins (1964) How To Murder Your Wife (1965) The Flim-Flam Man (1967) How To Save A Marriage and Ruin Your Life (1968) Justine, Changes, The Monk (all 1969) Squeeze A Flower (1970) Rabbit, Run (1970) Once Upon A Dead Man (1971) The Late Liz (1971) Pickup on 101 (1972) Where Eagles Fly (1972) Charlie and the Great Balloon Chase (1978) The Comedy Company (1978) Marriage is Alive and Well (1979) Charlie's Balloon (1981) The Fox and the Hound (1981) Dead and Buried (1981) (by no means an exhaustive list) His one regret was that he was not asked to reprise his role in the movie version of The Sunshine Boys.

Television also saw much of Albertson's talent. He had recurring or starring roles on:

The Burns and Allen Show The Thin Man Ensign O'Toole Chico and the Man, for which he won an Emmy, making him one of the few entertainers to win the triple crown of entertainment (Tony, Oscar, and Emmy. The tragedy of his success was this his co-star, Freddie Prinze, killed himself during a bout of depression in what is called "an accidental shooting." Grandpa goes to Washington He also guest starred on numerous shows, including: