Hovey was born in Mount Vernon, Indiana. After receiving an education in the public schools, he held a variety of jobs including bricklayer, school teacher, and lawyer. He received a commission in the U.S. Army during the Mexican-American War, but saw no action. After the war, he filled a vacant position as an Indiana state Supreme Court justice and served as a U.S. District Attorney when the Civil War started.

He was appointed Colonel of the 24th Indiana Volunteer Regiment and fought in the Battle of Shiloh under General Lew Wallace. In April, 1862, he was appointed brigadier general of volunteers and assumed command of the District of East Arkansas. He commanded the 12th Division, XIII Corps during the Vicksburg Campaign. Hovey took a prominent part in the Battle of Champion Hill and won the praise of General Ulysses S. Grant for his services there. He led the division through the Siege of Vicksburg. He commanded the 1st Division, XXIII Corps during the first part of the Atlanta Campaign until the division was discontinued in August of 1864. He then commanded the District of Indiana. At the wishes of Governor Oliver P. Morton, Hovey raised 10,000 new recruits to serve in the Union Army. He received a brevet promotion to major general in 1864 and resigned from the army the following year.

After the end of the Civil War, he was sent as a U.S. Minister to Peru. After serving in Peru for five years, he returned to his hometown of Mount Vernon and practiced law. He turned down a nomination for governor, but won election as Congressman in 1886. In 1888, he accepted a second nomination for governor and won the election. Hovey died in office in 1891 in Indianapolis, Indiana, and is buried in the Bellefontaine Cemetery near Mount Vernon.

Sources Civil War Indiana biography of Hovey Biography of Hovey Biography of U.S. Congressmen Indiana Historical Bureau: Biography and portrait

Preceded by: Isaac P. Gray Governor of Indiana January 14, 1888 - November 23, 1891 Succeeded by: Ira Joy Chase