Jones was born in Charlottesville, Virginia. At West Point, he was nicknamed "Rum" Jones for his fondness for alcohol. He graduated in 1841, ranking 39th in a class of 52. Nineteen of his classmates would become Civil War generals, including John F. Reynolds, Nathaniel Lyon, Robert S. Garnett, Richard B. Garnett, Amiel W. Whipple, and Israel B. Richardson, all of whom would also die in combat.

Assigned as a second lieutenant to the 7th Infantry, he was on frontier duty before returning to the U.S. Military Academy in 1845 as an assistant instructor of infantry tactics, serving until 1852. From 1854 until 1855, he served on a board commissioned to revise rifle and light artillery military tactics, and received a promotion to captain on March 3, 1855. Jones performed garrison duty at various forts across the country for a short period before participating in the Utah War from 1858 until 1860.

With the outbreak of the Civil War and Virginia's secession, Jones resigned his commission in the United States Army on May 27, 1861 to enter the Confederate service as a captain of artillery. He was appointed a colonel of infantry and served in what became the Army of Northern Virginia. He participated as a staff officer in Stonewall Jackson's Shenandoah Valley Campaign of 1862, Front Royal, the Seven Days Battles, the Second Battle of Manassas, Fredericksburg, and Chancellorsville. In May, Jones was promoted to brigadier general in Edward "Allegheny" Johnson's division to replace the incompetent John R. Jones (not related). During Johnson's assault on Culp's Hill at Gettysburg, Jones suffered a severe wound to his head that put him out of action. In August, Robert E. Lee called Jones "a good commander" when he assigned him to lead the brigade vacated by William "Extra Billy" Smith, who had returned to Virginia as governor. Late in the year, Jones was wounded again during the Mine Run Campaign.

In the early days of the Overland Campaign of 1864, Jones was killed at the Wilderness while attempting to rally his wavering men. His brigade had taken a position on the south side of the Orange Turnpike. Shortly before 1:00 p.m., it was hit hard by Roy Stone's Union brigade, which began pushing back the Confederates, many in disorder. Jones and his aide-de-camp Robert Early were killed while desperately trying to restore order. Jones' body was eventually returned home and buried in Maplewood Cemetery in Charlottesville. Lt. Gen. Richard S. Ewell wrote of Jones' death in his official report on the Wilderness, "I considered his loss an irreparable one to his brigade."

Jones' sister was married to famed Confederate general A.P. Hill's older brother Thomas.