Born in the Bronx, New York, Reiner was educated at the School of Foreign Service at Georgetown University and served in the United States Army during World War II. He later performed in several Broadway musicals, including Inside U.S.A., and Alive and Kicking , and had the lead role in Call Me Mister. In 1950, he was cast by Max Leibman in Sid Caesar's Your Show of Shows, and worked alongside writers such as Mel Brooks, Neil Simon and Woody Allen. He also worked on Caesar's Hour.

Reiner was frequently seen or heard playing the straight man to Mel Brooks' "2000 Year Old Man" character.

In 1961, Reiner created The Dick Van Dyke Show. In addition to usually writing the show, Reiner occasionally appeared as temperamental show host "Alan Brady", who ruthlessly browbeats his brother-in-law (played by the late Richard Deacon). The show ran from 1961 to 1966.

Reiner began his directing career on the Van Dyke show. His first feature was an adaptation of the play Enter Laughing (1967). Probably the best-known film of his early directing career was the cult comedy Where's Poppa (1970), starring George Segal and Ruth Gordon.

Reiner played a large role in the early career of Steve Martin, by directing and co-writing four films for the comedian; The Jerk in 1979, Dead Men Don't Wear Plaid in 1982, The Man with Two Brains in 1983, and All of Me in 1984. In 2000 Reiner was honored with the Mark Twain Prize for American Humor. In 2001, he played the character of Saul Bloom in Steven Soderbergh's Ocean's Eleven and its 2004 sequel. In 2004 he voiced the lion Sarmoti in the animated TV series Father of the Pride.