Born in San Francisco, California, he graduated from high school in Los Angeles and spent three years in the Army Corps of Engineers, where he took up boxing. After leaving the army, he enrolled in Los Angeles City College and the Los Angeles Academy of Dramatic Art. He had a small part in When Worlds Collide, then took small roles in such films as The All American, Brigadoon, Ten North Frederick and The Sound and the Fury. In 1957, Whitman was seriously considered for the role of Bart Maverick in the television series Maverick when the studio realized that they needed another Maverick to rotate as the series lead with James Garner, who closely resembled Whitman, but Jack Kelly wound up with the part. Whitman's first starring role was in Murder, Inc. in 1960.

In 1961, Whitman was nominated for an Academy Award for Best Actor for his role as a child molester in The Mark. He has since appeared in starring and supporting roles in many films, including Francis of Assisi, The Fiercest Heart, The Comancheros (sharing leading man status with John Wayne), Convicts Four, The Day and the Hour, Signpost to Murder, Shock Treatment, Rio Conchos, Those Magnificent Men in Their Flying Machines, Sands of the Kalahari, An American Dream, The Last Escape, The Decks Ran Red, The Invincible Six, Night of the Lepus, Shatter, Tony Saitta, and Guyana: Crime of the Century. Whitman had a memorable foray into television for a single season in 1967 as heroic Marshal Jim Crown in the lavish and critically acclaimed 90-minute weekly western series Cimarron Strip, which was also widely noted for its thrilling theme music.