His early years were filled with dreams of the stage. After numerous failed serious plays, Berman found himself with The Compass Players, an improv comedy troupe, where he found his first success. This troupe would later became The Second City.

He left the group in 1957 and created his own comedy style. Contrasting with the frenetic, ad libbed style popular at the time, Berman's act was perfectly timed and poignant, while remaining improvised. This would be expected, given his theater background.

His preferred mode of delivery was seated on a high stool at center stage, as opposed to his contemporaries, who tended to stalk the stage or prop themselves up on a piano or microphone stand. His honest, biting, satiric comedy style soon found its niche.

His 1959 debut album won the first non-music Grammy ever, and the cover of that album cemented his image as a skinny guy sitting on a high chair, cigarette in one hand and a microphone in the other. He went on to appear on the Ed Sullivan Show over 20 times, The Jack Paar Show, and numerous other variety and TV specials throughout the 1960s.

During the 1970s and 1980s, Berman stayed out of the limelight following undisclosed personal issues and the death of his son. By 1989, he had returned to comedy by taking small roles. In 1995, Berman released a new live comedy album.

Since then, he has been part of numerous projects, including Curb Your Enthusiasm, Friends, Arli$$, King of Queens, The Bernie Mac Show and Meet The Fockers.

Since 1982, Berman has been teaching in the Master of Professional Writing Program at The University of Southern California, in the subject of Writing Humor, Literary and Dramatic. He has written three books, and continues to write and act.