His best-known work is The Phil Silvers Show, a 1950s sitcom set on a US Army post in which he played Sergeant Bilko; the show was also often referred to by this name. He won a Tony Award for Top Banana in 1952 (it was turned into a film in 1954). The show's chief writer, Nat Hiken, was TV's first writer-producer, and Hiken helped set a high comic tone for the show through his inventive plots and snappy comedic repartee for the characters.

Born Philip Silversmith in Brooklyn, New York, Silvers was the youngest of eight children in a Russian Jewish family. His father was one of the workers on the early New York skyscrapers. Silvers started entertaining at age 11, when he would sing in theaters when the projector broke down (a common occurrence in those days). Two years later, he left school to sing professionally, before appearing in vaudeville as a stooge.

Silvers then landed work in short films, burlesque houses, and on Broadway, where he made his debut in the short-lived show Yokel Boy. The critics raved about Silvers, who was hailed as the bright spot in the mediocre play. He then wrote the revue High Kickers, until he went to Hollywood to star in films.

He made his film debut in Hit Parade of 1941 (1940) (his previous appearance as a pitch man in Strike Up the Band was cut). Over the next two decades, he appeared in character roles for MGM, Columbia, and 20th Century Fox, in such films as Lady Be Good, Coney Island, Cover Girl, and Summer Stock. When the studio system started collapsing, he then turned to television and more stardom in the role of Sgt. Ernie Bilko.

Throughout the 1960's, he appeared in films such as It's a Mad Mad Mad Mad World and A Funny Thing Happened on the Way to the Forum. He also guested on various variety shows such as The Carol Burnett Show, Rowan & Martin's Laugh-In, and The Dean Martin Show.

He famously starred as a guest in one of the famed British Carry On films, Follow That Camel (1967), as a Sergeant Bilko character in a spoof of the Foreign Legion films. Peter Rogers employed him to ensure the Carry On films' success in America.

His salary was £30,000, the largest Carry On salary ever, only later met by the appearance of Elke Sommer in Carry On Behind (1975). Despite his appearance in the film, he didn't ensure the film's success on either side of the Atlantic.

Famed voice actor Daws Butler employed an impression of Silvers as the voice of the Hanna-Barbera cartoon character Hokey Wolf and also used the same voice in numerous cartoons for Jay Ward. Furthermore, the premise of The Phil Silvers Show was the basis for the Hanna-Barbera cartoon Top Cat.

Silvers was very ill in the last few years of his life, even though he continued work into the early 1980s in film and TV, including a cameo appearance on Happy Days as the father of "Jenny Piccolo" (played by his real-life daughter Cathy Silvers). He died in 1985 of a heart attack at the age of 74.

In a 2005 poll to find The Comedian's Comedian, he was voted #42 on the list of the top 50 comedy acts ever by fellow comedians and comedy insiders.