Born Henry DeWitt Carey II in The Bronx, New York, he was the son of a prominent lawyer and judge. He attended Hamilton Military Academy then studied law at New York University.

Although Carey,one of Hollywood's finest character actors of the sound era, received an Oscar nomination for his role as the President of the Senate in the 1939 film, Mr. Smith Goes to Washington, he is best rememberd as one of the first stars of the Western film genre. He married actress Olive Fuller Golden (1896-1988) and they purchased a large ranch in Saugus, California, north of Los Angeles. Their son, Harry Carey, Jr. would become a character actor, also most famous for his roles in Westerns. They appeared together in the 1948 film, Red River, which was filmed late in 1946 and went unreleased for almost two years.

Carey also performed on the stage, debuting on Broadway in 1940.

A smoker, Harry Carey died in 1947 from a combination of lung cancer, emphysema and coronary thrombosis, at the age of 69. He was interred in Woodlawn Cemetery in the family mausoleum in The Bronx, New York. In 1976, he was inducted into the Western Performers Hall of Fame at the National Cowboy & Western Heritage Museum in Oklahoma City, Oklahoma.

For his contribution to the motion picture industry, Harry Carey has a star on the Hollywood Walk of Fame at 1521 Vine Street.