Kahn was born in Boston, Massachusetts, as Madeline Gail Wolfson to a Jewish family. Her mother Paula was 17 when Madeline was born. Although Kahn's parents were high school sweethearts, they divorced when Madeline was 2. After the divorce was finalized, Kahn and her mother moved to New York City. A few years later, her mother remarried and this union gave Kahn two half-siblings, Jeffrey and Robyn. In 1948, Kahn was sent to a progressive boarding school in Pennsylvania, where she stayed until 1952. During that time, her mother pursued her ambition as an actress. Kahn soon began acting herself and performed in a number of school productions. In 1960, she graduated from the Martin Van Buren High School in Queens, New York, where she earned a drama scholarship to Hofstra University. At Hofstra, she studied music, drama and speech therapy and also performed in several campus productions. After changing her major a number of times, Kahn graduated in 1964 with a degree in speech therapy.

Kahn began auditioning for professional acting roles shortly after her graduation from Hofstra; on the side, she briefly taught public school in Levittown, New York. Just before adopting the professional name Madeline Kahn (Kahn was her stepfather's last name), she made her stage debut as a chorus girl in a revival of Kiss Me, Kate, which led her to join the Actors' Equity. Her part in the flop How Now, Dow Jones was written out before the 1967 show reached Broadway, as did her role as "Miss Whipple" in the original production of Promises, Promises. But she earned her first break on Broadway with New Faces of 1968. That same year, she performed her first professional lead in a special concert performance of the operetta Candide in honor of Leonard Bernstein's 50th birthday. In 1969, she appeared off-Broadway in the revue Promenade.

She appeared in two Broadway musicals in the 1970s: a featured role in Richard Rodgers' 1970 Noah's Ark-themed show Two by Two (her silly waltz "The Golden Ram," capped by a high C, can be heard on the show's cast album) and a leading lady turn as Lily Garland in 1978's On the Twentieth Century. She left (or was fired from) the latter show early in its run, yielding the role to her understudy, Judy Kaye, whose career it launched. She also starred in a 1977 Town Hall revival of She Loves Me (opposite Barry Bostwick and original London cast member Rita Moreno).

She debuted in the movies that same year with a role in De Düva: The Dove. Her most famous roles followed in the 1970s: she appeared in What's Up, Doc? (1972), Paper Moon (1973), Young Frankenstein (1974), Blazing Saddles (1974) and High Anxiety (1977). The final three films were all directed by Mel Brooks, who many Hollywood observers claimed was able to bring out the best of Kahn's comic talents. (Their last collaboration would be 1981's History of the World, Part I.) For her work in Paper Moon and Blazing Saddles, the young comedienne received nominations for Academy Award for Best Supporting Actress.

Kahn's roles were primarily comedic rather than dramatic, though the 1970s found her originating roles in two plays that had both elements: 1974's In the Boom Boom Room and 1977's Marco Polo Sings a Solo. After her success in Brooks' films, she played in a number of less successful films in the 1980s (perhaps most memorably as Mrs. White in 1985's terminally silly Clue: The Movie).

Late in her career, she returned to the stage, first in Judy Holliday's role in a 1989 revival of Born Yesterday, then winning a Tony Award as "Dr. Gorgeous" in 1993's The Sisters Rosensweig, a play by Wendy Wasserstein. She played Angela Lansbury's role in a concert revival of Anyone Can Whistle that was released on CD.

In the final years of her life, she played a regular role on the sitcom Cosby and voiced Gypsy the moth in A Bug's Life. She also received some of the best reviews of her career for her Chekhovian turn in the 1999 independent movie Judy Berlin, the last film she ever made.

In the early 1990s, Kahn recorded a voice for the animated movie The Magic 7; along with John Candy, she will be one of two deceased actors with voices in that movie.

Kahn succumbed to ovarian cancer on December 3, 1999, at the age of 57. She was survived by her husband, John Hansbury; mother, Paula Kahn; brother, Jeffrey Kahn; and niece, Eliza Kahn.