Born in Philadelphia, Pennsylvania, the son of Irish immigrants, he attended St. Joseph's College and the Collegiate Institute at Benicia, California. After being admitted to the California bar in 1865, he became District Attorney for Solano County and then served in the California State Assembly for two years.

McKenna was elected to the United States House of Representatives in 1885 and served for four terms. He was appointed to the Ninth Circuit Court of Appeals in 1892 by President Benjamin Harrison.

In 1897 he was appointed Attorney General of the United States by President William McKinley, and served in that capacity until 1898. He was then appointed an Associate Justice of the Supreme Court of the United States to succeed Justice Stephen J. Field. McKenna faced opposition to his appointment in the Senate, but his supporters there enabled him to win confirmation in January 1898. Conscious of his limited credentials, McKenna took courses at Columbia Law School for several months to improve his legal education before taking his seat on the Court.

McKenna was known to be a centrist, and authored few dissents. His most noteworthy opinions are Hipolite Egg Co. v. United States, in which a unanimous Court upheld the Pure Food and Drug Act of 1906, and Hoke v. United States, which upheld the Mann Act. While McKenna was generally quite favorable to federal power, he voted with the majority in 1905's Lochner v. New York, which struck down a state maximum-hours law for bakery workers.

McKenna resigned from the Court in January 1925 at the suggestion of Chief Justice Taft. McKenna's ability to perform his duties had been diminished significantly by a stroke suffered ten years earlier, and by the end of his tenure McKenna could not be counted on to write coherent opinions.

McKenna was married to Amanda Borneman in 1869; they had three daughters and one son. He died in 1926 in Washington, D.C. and lies buried in Mount Olivet Cemetery.