Chandler was born in Bedford, New Hampshire, the nephew of John Chandler and Thomas Chandler, the grandfather of Frederick Hale, and the great-great-granduncle of Rodney Dennis Chandler.

After receiving an education in the common schools, Chandler taught school for one winter while also managing the family farm. His father offered him a choice between a college education or $1,000, which was a substantial amount of money at that time. Chandler took the money, and in 1833 moved to Detroit, which was then the captital of Michigan Territory. He established a successful general merchandise store with his brother-in-law, and also engaged in land speculation and banking. Through frugality and determination, he quickly became quite wealthy. In 1845, he was a part of the corporation that bought the state-charted Michigan State Bank. He became one of the first men in Detroit to earn $50,000 a year from his businesses.

He was a member of the Whig party and demonstrated considerable resolve to ensure the freedom to vote. He reportedly heard that some members of the local Democratic party were planning to form a crowd around a ballot box to prevent anyone from another party from voting. Chandler showed up at the poll carrying a cudgel and spent the entire election patrolling the area to ensure that anyone who wanted to vote was able to do so.

In 1851, Chandler's career in political office began when he defeated the popular General John R. Williams in the election for mayor of Detroit. While mayor, Chandler first encountered Ulysses S. Grant, who was then a Lieutenant in the U.S. Army. Grant sued Chandler for not clearing the ice and snow in front of his home after an officer had slipped and suffered a severe sprain. Chandler chose to defend himself in a trial by jury, and aggressively confronted the Army officers, accusing them "If you soldiers would keep sober, perhaps you would not fall on people's pavements and hurt your legs." Grant ultimately won the case, but had also worsened his reputation for drinking heavily. Chandler was fined only six cents and court costs of about $8. According to popular lore, Chandler later had Grant ticketed for driving too fast in a carriage.

In 1852, he was the Whigs' candidate for governor, but lost the election to incumbent Robert McClelland. He was active in leading anti-slavery Whigs into the formation of the Republican Party in Jackson, Michigan in 1854. He was a vigorous opponent of slavery and lent his assistance to the Underground Railroad. He was elected to the U.S. Senate as a Republican in 1857, taking the seat that had been held by Lewis Cass. Chandler attacked the 1857 Dred Scott U.S. Supreme Court decision which upheld the Fugitive Slave Law.

In 1858, Chandler opposed the admission of Kansas under the Lecompton constitution, which allowed slavery, and took an active part in debates over this issue. On February 11, 1861, Chandler wrote the famous so-called "blood letter" to Austin Blair, the Governor of Michigan. This letter contained the sentence, "Without a little blood-letting this Union will not, in my estimation, be worth a rush." The letter was quoted throughput the country, and Chandler defended his statement on the floor of the Senate. He was closely associated with Senators Benjamin F. Wade of Ohio and Lyman Trumbull of Illinois, whom Lincoln's secretary and biographer John Hay derisively referred to as the "Jacobin Club", alluding to the infamous extremists of the French Revolution. In July, 1861, Chandler, along with Wade, Trumbull and James Grimes, witnessed the First Battle of Bull Run, which was a disaster for the Union forces. At one point, Chandler came close to being captured by the Confederate Army.

As a Radical Republican, Chandler was critical of President Abraham Lincoln for not taking stronger action immediately against the southern states attempting to secede from the Union. He was also very critical of General George McClellan for not aggressively pursuing victory on the battlefield. Like other radical Republicans, he was also critical of Lincoln's Reconstruction plan. In 1868, he was active in the campaign to impeach President Andrew Johnson, whom he viewed as an incompetent willing to sacrifice all the gains made during the war through "soft" reconstruction.

Chandler was reelected in 1863 and again in 1869, serving from March 4, 1857, to March 3, 1875 in the 35th through the 43rd U.S. Congresses. He was defeated by Isaac P. Christiancy while seeking election for a fourth term in 1874, when the Michigan legislature deadlocked following a Democratic landslide in elections that year. Chandler served as the chairman of the Committee on Commerce from 1861 to 1875 and was responsible for funneling large amounts of federal funding into the developing Midwest.

He was appointed Secretary of the Interior by President Ulysses Grant in 1875 and served until 1877. The Interior Department included the Bureau of Indian Affairs, which was rife with corruption. Chandler fought his predecessor Delano on political patronage in the Department and as party boss, had no reformist tendencies. However, Chandler surprised many my moving quickly to uncover fraud and dismiss corrupt people in the Interior Department. When the next administration came to power, Chandler's was one of the few departments to receive compliments from the incoming staff.

Chandler managed Rutherford B. Hayes' successful 1876 campaign for the presidency, though Hayes declined to keep Chandler as Secretary of the Interior. In 1879, he was again elected to the Senate to fill the vacancy caused by the resignation of Isaac P. Christiancy, who had succeeded him just four years earlier. He served in the 45th and 46th Congresses from February 22, 1879, until his death.

Being considered as a possible presidential candidate, Chandler went to Chicago to deliver a political speech on October 31, 1879, and was found dead in his room on the following morning. He is interred at Elmwood Cemetery in Detroit. Chandler was a lifelong member of the Presbyterian Church.