Williams was born in Deep River, Connecticut. He graduated from Yale University with a law degree in 1831. His father, who died when Williams was eight years old, had left him a sizable inheritance, which he used between 1832 and 1836 for extensive travel in the United States and Europe. Upon his return, he settled in Detroit, Michigan, which was a booming frontier town in 1836. He established himself as a lawyer and married the daughter of a prominent family, Jane Hereford, with whom he produced five children, two of whom died as infants. Jane herself died young as well, in 1849, at the age of 30.

Williams had a variety of careers in Detroit. He was elected probate judge of Wayne County, Michigan; in 1842, president of the Bank of St. Clair; in 1843, the owner and editor of the Detroit Advertiser daily newspaper; from 1849 to 1853, postmaster of Detroit.

When Williams arrived in Detroit in 1836, he joined a company in the Michigan Militia and maintained a connection to the military activities of the city for years. In 1847, he was appointed lieutenant of a regiment destined for the Mexican War, but it arrived too late to see any action. He also served as the president of the state's military board and in 1859 was a major in the Detroit Light Guard.

At the outbreak of the Civil War in 1861, Williams was involved in training the first army volunteers in the state. He was promoted to brigadier general of volunteers on May 17, 1861. His first assignment after leaving the training camps was as a brigade commander in Nathaniel Banks's division of the Army of the Potomac, from October 1861 to March of 1862. He then assumed division command in the V Corps of the Army of the Potomac, as of March 13, 1862. This division was transferred to the Department of the Shenandoah from April to June of that year. Williams and Banks were sent to fight Stonewall Jackson in the Shenandoah Valley and were thoroughly outmaneuvered, allowing Jackson to bottle them up in the Valley with his much smaller force.

On June 26, Williams division was transferred to the Army of Virginia, under John Pope, for the Northern Virginia Campaign. In the Battle of Cedar Mountain, Banks's Corps was again up against Jackson, and was again defeated. Williams's division did not reach the Second Battle of Bull Run until after the battle was over.

Williams's division rejoined the Army of the Potomac as the 1st Division of the XII Corps and marched north in the Maryland Campaign to the Battle of Antietam. On the way, troops from the division found the famous Confederate "lost dispatch," Special Order 191, that revealed Robert E. Lee's plan for the campaign and gave Major General George B. McClellan key insights on how to defeat Lee's divided army. The division was heavily engaged at Sharpsburg, once again up against Jackson on the Confederate left flank. The corps commander, Joseph K. Mansfield, was killed early in the battle and Williams assumed temporary command. The corps suffered 25asualties in assaulting Jackson and was forced to withdraw. Maj. Gen. Henry W. Slocum replaced Williams as permanent corps commander immediately after the battle.

Williams missed the next major battle for the Army of the Potomac, the Battle of Fredericksburg, because it was engaged in defending the Potomac River. In the Battle of Chancellorsville, on May 2, 1863, Stonewall Jackson's corps executed a surprise flanking movement and smashed into the right flank of the Army of the Potomac, severely damaging the unsuspecting XI Corps. The neighboring division, under Williams, entrenched hastily and was able to stop the Confederate advance before it overran the entire army, but it suffered 1,500 casualties in the process.

In the Battle of Gettysburg, William's division arrived on the battlefield late in the afternoon of July 1, 1863, and occupied Benner's Hill, east of the town of Gettysburg. On July 2, the XII Corps took up positions on Culp's Hill, the right flank of the Union line. At this point, due to a command misunderstanding, Henry Slocum believed that he was in command of the "Right Wing" of the army, consisting of the XI and XII Corps. Therefore, Williams assumed temporary command of the XII Corps and controlled it for the rest of the battle.

On the afternoon of July 2, a massive attack by James Longstreet on the Union's left flank caused army commander George G. Meade to order Williams to transfer his entire corps to reinforce the left, in the vicinity of Little Round Top. Williams convinced Meade of the importance of Culp's Hill and managed to retain one brigade, under Brigadier General George S. Greene, in their defensive positions. In an heroic defense, Greene and his brigade withstood the assault of Edward "Allegheny" Johnson's Confederate division (the "Stonewall Division") throughout the night, until the remaining brigades of the XII Corps returned. Early on July 3, Williams launched an attack against the Confederates who had occupied some of the entrenchments on the hill and after a seven-hour battle, regained his original line. Unfortunately for Williams, General Slocum was late in writing his official report of the battle, and Meade submitted his report for the army without acknowledging the critical contributions that Williams and XII Corps made to the Union defense.

In September 1863, the Union army in Tennessee was defeated at the Battle of Chickamauga and two corps were sent west to help them as they were besieged in Chattanooga - the XI and XII Corps, combined due to their small sizes into a new XX Corps, Army of the Cumberland. Williams's division did not reach Chattanooga, but guarded railroads in eastern Tennessee. However, it did join William T. Sherman in the Atlanta Campaign and fought with distinction in a number of battles, particularly the Battle of Resaca. Williams was wounded in the arm at the Battle of New Hope Church on May 26, 1864. His division followed Sherman through his March to the Sea and the Carolinas Campaign. During this period, Williams received a brevet promotion to major general on January 12, 1865.

After the war, William served as a military administrator in southern Arkansas until he left the service on January 15, 1866. He returned to Michigan, but faced financial difficulties that forced him to take a post as the U.S. Minister to San Salvador, a position in which he served until 1869. He ran for governor of Michigan in 1870, but was unsuccessful. He served as the U.S. Representative from Michigan from March 4, 1875, to December 21, 1878. Williams died in Washington, D.C., as a congressman, and is buried in Elmwood Cemetery in Detroit.

Williams was a general who never received much public recognition. Despite fighting in important commands, he remained a brigadier general throughout most of the war. There were three reasons likely for the situation: first, he was not a West Point graduate, and the "old boys network" was as effective in the 19th century as it is today; second, during the formative months of the Army of the Potomac, Williams was stationed in the Shenandoah Valley, which denied him familiarity to the high command when reputations were being established; third, Williams was never comfortable mastering the common practice of promoting himself to the public with the help of friendly newspaper correspondents. Williams did, however, communicate well with his family and the letters he wrote throughout the war were saved and published posthumously in 1959 as the well regarded book, From the Cannon's Mouth: The Civil War Letters of General Alpheus S. Williams.

A powerful equestrian memorial to Williams by sculptor Henry Shrady stands in Belle Isle, in the Detroit River, next to his home town of Detroit.