After growing up in Baltimore, he attended American University in Washington, D.C. before moving to Los Angeles to work as an actor and writer. His first writing work was for variety shows such as The Marty Feldman Comedy Machine, The Lohman and Barkley Show, The Tim Conway Show, and The Carol Burnett Show.

After some success as a screenwriter (Silent Movie, 1976, High Anxiety, 1977, ...And Justice for All, 1979), he began his career as a director with Diner (1982), for which he had also written the script and which earned him a Best Screenplay Oscar nomination. Diner was the first of a series of films set in the Baltimore of Levinson's youth. The other films in this series were Tin Men (1987), starring Richard Dreyfuss and Danny DeVito, and the turn-of-the-century immigrant family saga Avalon (which featured Elijah Wood in one of his earliest screen appearances), as well as the more recent Liberty Heights (1999). All four movies were written and directed by Barry Levinson himself; for the last two he also acted as producer.

His biggest hit, both critically and financially, was Rain Man (1988) with Dustin Hoffman and Tom Cruise. The film won four Academy Awards including Best Director for Levinson. Other notable films in his directing career were The Natural (1984), Good Morning, Vietnam (1987) with Robin Williams, and Bugsy (1991) with Warren Beatty.

He directed Dustin Hoffman again in Wag the Dog (1997), a political comedy about a war staged in a film studio. He was also an uncredited co-writer on Dustin Hoffman's transvestite comedy Tootsie (1982).

Apart from producing many of his own films, he has also been producer or executive producer for such major productions as The Perfect Storm (directed by Wolfgang Petersen, 2000), Analyze That (2002, starring Robert de Niro as neurotic mafia boss and Billy Crystal as his therapist), and Possession (2002, based on the bestselling novel by A. S. Byatt). He was also executive producer for the television series Homicide: Life on the Street (which ran on NBC from 1993-1999) and the HBO prison drama Oz.

Levinson published his first novel, Sixty-Six (ISBN 076791533X), in 2003. Like several of his films, it is semi-autobiographical and set in Baltimore in the early 1960s.

Levinson married his writing collaborator Valerie Curtin in 1975. They would divorce seven years later. He later married Dianna Rhodes whom he met in Baltimore while filming Diner.