Sherman was born in Utica, New York, gaining his early education through private schools. Graduating from Hamilton College in 1878, he studied law for two years, then was admitted to the bar in 1880. He is a member of the Sigma Phi fraternity.

Nicknamed "Sunny Jim" for his positive disposition, Sherman practiced law at the Utica firm of Cockinham & Martin and was married to Carrie Babcock in 1881. He also served as president of the Utica Trust & Deposit Co. and of the New Hartford Canning Co. during this period before becoming mayor of Utica in 1884.

Elected to Congress in 1886 as a Republican, Sherman served four years before losing his seat following the 1890 election to Henry W. Bentley. Sherman would return two years later to win the first of eight consecutive elections for the seat, helping to establish his presence in the nation's capital.

In 1898, he briefly became a candidate for Speaker of the House, before being offered the office of General Appraiser of the Port of New York one year later by U.S. President William McKinley. Sherman declined the position after vocal opposition from his Utica constituents.

At the time of his 1908 nomination as the Republican candidate for Vice President on the ticket with William Howard Taft, Sherman was Chairman of the House Committee on Indian Affairs. During the ensuing campaign, Sherman endured a blistering attack from Los Angeles attorney Edmund Burke, who had claimed he was a former partner of his in the New Mexico Lumber and Development Company. Burke alleged that Sherman had used his influential position to further a land grab deal, but his accusations would not prove to be damaging in the election.

Sherman had also been diagnosed with gallstones during this period, and was told that kidney trouble was causing his problems. By following a strict diet, he was able to manage his lingering condition after taking office on March 4, 1909. However, his strong work ethic played a major role in the deterioration of his health over the course of the 1912 presidential campaign.

In June 1912, Sherman became the first incumbent Vice President to be renominated since Richard M. Johnson in 1840. The ensuing campaign saw Taft in a stiff three-way battle with Democrat Woodrow Wilson and former U.S. President Theodore Roosevelt, the latter candidate having previously been in a nasty feud with Sherman over control of the New York Republican party.

Sherman died in Utica, Oneida County, New York a few days prior to the election. The Republican National Committee replaced Sherman on the ballot with Columbia University President Nicholas M. Butler. However, Democrats Woodrow Wilson and Thomas R. Marshall won in an electoral vote landslide long before this. James S. Sherman was interred in Forest Hill Cemetery.

This article incorporates facts obtained from the public domain Biographical Directory of the United States Congress.