He began his career in newspapers, writing for Stars and Stripes in the European Theater during World War II. During that time he was one of the first American journalists to visit the Nazi death camps as World War II wound down, and one of the first to write about the death camps.

Rooney also was a freelance writer and a television script writer before joining 60 Minutes. In the early 1950s he was a writer for Arthur Godfrey when Godfrey was at the peak of his powers on CBS radio and TV though Rooney later moved on to other projects.

Though originally a regular correspondent, Rooney now has his own "end-of-show" segment, "A Few Minutes with Andy Rooney," in which he offers a light-hearted editorial on a trivial everyday issue, such as the cost of groceries, annoying relatives, or faulty Christmas presents. Rooney has always considered himself a writer who appears on television.

According to CBS News's biography of him,

Rooney wrote his first television essay, a longer-length precursor of the type he does on 60 Minutes, in 1964, “An Essay on Doors.” From 1962 to 1968, he collaborated with the late CBS News Correspondent Harry Reasoner—Rooney writing and producing, Reasoner narrating—on such notable CBS News specials as “An Essay on Bridges” (1965), “An Essay on Hotels” (1966), “An Essay on Women” (1967), and “The Strange Case of the English Language” (1968). “An Essay on War” (1971) won Rooney his third Writers Guild Award. In 1968, he wrote two CBS News specials in the series “Of Black America.” His script for “Black History: Lost, Stolen or Strayed” won him his first Emmy. His shorter television essays have been archived in numerous books, such as Common Nonsense, which came out in 2002, and Years of Minutes, released in 2003. He also has a regular syndicated newspaper column that runs in many newspapers in the United States. He has won three Emmy Awards for his essays, which now number more than 800. He was also awarded a Lifetime Achievement Emmy.

Rooney is popularly thought to be an atheist based on a series of comments he made regarding Mel Gibson's film The Passion of the Christ. In public comments, he has described himself as an agnostic. He has admitted on Larry King Live to having a liberal bias.

Rooney attended the Albany Academy in Albany as a boy, and later attended Colgate University in Hamilton, New York, until he was drafted into the Army in 1941. He has four children, including a daughter, Emily Rooney, who is also a journalist and former ABC News producer, who currently hosts a nightly Boston-area public affairs program, Greater Boston, on WGBH. His son Brian Rooney has been a corespondent for ABC since the 1980's. His wife of 62 years, Marguerite, died in 2004.

In the fall of 2005, Andy Rooney's support for medical doctors that "do not advertise" (aired on national television) emphasized the foundation of ethics in doctor-patient relations.