The son of Nikonar Keitel and Maritska LeCose, Jewish immigrants from Poland and Romania, he grew up in the Brighton Beach section of Brooklyn. At the age of 16, Keitel decided to join the United States Marine Corps, a decision which took him to Lebanon. After his return to the United States, he floundered for a while in mediocre jobs before beginning his acting career.

Keitel studied under both Stella Adler and Lee Strasberg, eventually landing roles in some off-Broadway productions. During this time, Keitel met another struggling filmmaker named Martin Scorsese and gained a part in Scorsese's student production. Since then both Scorsese and Keitel have worked together on numerous projects. Keitel had the starring role in Scorsese's Mean Streets but this proved to be Robert De Niro's breakthrough film.

Originally, Keitel was to have played the role of Captain Willard in Francis Ford Coppola's Apocalypse Now. However, he was fired early in the production and replaced by Martin Sheen. After this, it was many years before he would be able to get anything other than minor roles. At the end of the 1970s Keitel was mostly working in European films for directors such as Ridley Scott, usually in sinister character parts. Keitel was also fired from the set of Eyes Wide Shut, Stanley Kubrick's last film, halfway through production. He was replaced with Sydney Pollock.

Throughout the 1980s, Keitel continued to find plenty of work on both stage and screen, but it was usually in the stereotypical role of a thug. This role reached its apotheosis when Keitel starred in Quentin Tarantino's Reservoir Dogs in 1992, where his performance as Mr. White made him a star. Ridley Scott also helped Keitel by casting him as the sympathetic policeman in Thelma and Louise. Since then, Keitel has chosen his roles with care, seeking to change his image and show off a broader acting range. His decision to co-star in Jane Campion's The Piano marks the approximate beginning of this phase of Keitel's career. He has also shown a willingness to help other upstart filmmakers by appearing in their first feature film. He has done this not only for Martin Scorsese and Quentin Tarantino, but also Ridley Scott (The Duellists), Paul Schrader (Blue Collar), James Toback (Fingers), and Tony Bui (Yellow Lotus).