Born in Annapolis, Johnson was the son of a distinguished Maryland lawyer and politician, John Johnson (1770 - 1824). He graduated from St. John's College in 1812 and then studied law. He was admitted to the bar in 1815, and then moved to Baltimore, where he became a legal colleague of Luther Martin, William Pinkney and Roger B. Taney. From 1821 until 1825 he served in the Maryland State Senate and then returned to practice law for two decades. From 1845 to 1849, he represented Maryland in the United States Senate as a Whig, and from March 1849 until July 1850 he was Attorney General of the United States under President Zachary Taylor. He resigned that position when Millard Fillmore took office.

A conservative Democrat, he supported Stephen A. Douglas in the presidential election of 1856. He represented the slave-owning defendant in the infamous 1857 case Dred Scott v. Sandford. Personally opposed to slavery, he was reportedly a key figure in the effort to keep Maryland from seceding from the Union during the American Civil War.

He served as a Maryland delegate to the 1861 peace convention and from 1861 to 1862 served in the Maryland House of Delegates. After the capture of New Orleans, he was commissioned by President Abraham Lincoln to revise the decisions of the military commandant, General B.F. Butler, in regard to foreign governments, and reversed all those decisions to the entire satisfaction of the administration. After the war, representing the riven points of view held by his fellow statesmen, Johnson argued for a gentler Reconstruction effort than that advocated by the Radical Republicans.

In 1863 he again took a seat in the United States Senate, serving through 1868. In 1866, he was a delegate to the National Union Convention which attempted to build support for President Johnson. Senator Johnson's report on the proceedings of the convention was entered into the record of President Johnson's impeachment trial. In 1868 he was appointed minister to the United Kingdom and soon after his arrival in England negotiated the Johnson-Clarendon Treaty for the settlement of disputes arising out of the Civil War; this, however, the Senate refused to ratify, and he returned home on the accession of General Ulysses S. Grant to the presidency. Again resuming his legal practice, he was engaged by the government in the prosecution of cases against the Ku Klux Klan as well as work compiling the reports of the decisions of the Maryland Court of Appeals.

He died in 1876 in Annapolis and is buried in Greenmount Cemetery at Baltimore.