He was born in Cedarhurst, Long Island and grew up there and in Manhattan. He graduated from Dwight Preparatory School in New York and entered Colgate University in the fall of 1939 as a member of the class of 1943. As with many schools at that time, Colgate offered its students the opportunity to accelerate their studies to facilitate their enlistment in the military. Peyser graduated from Colgate in December of 1942 and enlisted in the U.S. Army as a private in 1943.

Peyser entered the European theater about three months after D-Day. He served in the infantry as a "replacement" (i.e. assigned to a specific unit only after arrival in the theater to fill in ranks depleted by casualties). He saw action with the 99th Infantry Division in the Battle of the Bulge, Remagen Bridge Engagement and Huertgen Forest. After V-E Day, Sergeant Peyser participated in the occupation of a small town in Germany. He was discharged in 1946 and accepted a commission as second lieutenant with the Seventh Regiment of the New York National Guard, and later held the rank of captain.

Returning to New York, Peyser worked for the Manhattan Life insurance company in the agency managed by his father, Percy A. Peyser. A few years later, he joined the Mutual of New York life insurance company as an agent. He later managed offices in White Plains, New York and Manhattan.

On December 23, 1949, he married Marguerite Richards. A native of Monroe, Louisiana, the new Mrs. Peyser had grown up there and in Baltimore, and had moved to New York to attend the Parsons School of Design. In 1951, the Peysers moved to Irvington, Westchester County, New York. They had five children - Penelope (born 1951), Carolyn (born 1952), Peter (born 1954), James (born 1956) and Thomas (born 1962).

Peyser's political career began in 1962 when he ran for Mayor of Irvington. A community of 5,000 people, Irvington was governed by a part-time Board of Trustees and Mayor. The Mayor was paid $100 a month for his efforts. During four terms as Mayor, Peyser emphasized the need to upgrade the community's parks and its recreation department. He also presided over the upgrade of the village’s water system.

In 1969, Peyser announced a dark-horse candidacy for Congress as a Republican. The 25th District of New York at that time covered the Western half of lower Westchester County and three towns in Rockland County. At the time of his announcement, the incumbent Congressman from the area was Richard L. Ottinger, a popular Democrat. Later in 1969, Ottinger announced his candidacy for the U.S. Senate, seeking to oust Republican Sen. Charles Goodell, who had been appointed by Governor Nelson Rockefeller to fill Robert F. Kennedy’s seat after the latter's assassination in 1968. After Ottinger's announcement, three other Republicans, all with higher public profiles than Peyser, joined the race for the GOP nomination. Peyser won the June primary and went on to win the general election against William Dretzin. Peyser’s slogan was "Nixon Picks Him."

For the 1972 election, the district was redrawn because of the 1970 census. Now labeled the 23rd , it was 1/3 the northern Bronx, 1/3 the City of Yonkers, and 1/3 suburban communities along the Hudson River. That year, while Nixon carried the district comfortably, Peyser eked out a 1200 vote margin over former Rep. Richard L. Ottinger who sought to return to Congress after his defeat for the Senate in 1970.

During his three terms in the House as a Republican, Peyser made a name for himself as a consumer activist on the Committee on Agriculture, as assignment usually not sought by New York Members. His major efforts were directed to the Committee on Education and Labor where he fought for improved student loan programs, aid to elementary and secondary education and better worker training programs. He played an important role on the "ERISA Task Force" set up by House leaders to develop the landmark legislation that governs employee benefit and retirement plans. Peyser was a staunch Nixon supporter and backed his Vietnam War policies. He was one of the last Members of the New York Congressional delegation to call for the President’s impeachment, doing so only days before Nixon announced his resignation.

In 1976, Peyser launched an ill-fated attempt to wrest the GOP nomination for the U.S. Senate away from Senator James Buckley. Buckley had won the three-way 1970 Senate race on the Conservative Party line, but aligned himself with the Republican caucus in the Senate. Buckley gained an agreement with then-Vice President Nelson Rockefeller that in exchange for Buckley not endorsing Ronald Reagan's challenge to President Gerald Ford for the GOP nomination, Rockefeller would send a signal to New York Republicans to deny Peyser the resources to conduct his campaign. The GOP state committee sued the Peyser campaign to keep it off the September primary ballot, but the petition signatures gained by the Congressman withstood the test. However, Buckley won the primary in a landslide. He went on to defeat in the general election at the hands of Daniel Patrick Moynihan.

Jilted by his party, Peyser announced in early 1977 that he was changing to the Democratic party. Shortly thereafter, his former congressional colleague, Governor Hugh Carey, nominated Peyser to be Chairman of the New York State Public Utility Commission, perhaps the most powerful regulatory position in New York State at the time. The Republican-controlled Senate, from which confirmation was required, immediately objected to the nomination as an example of cronyism, citing Peyser’s lack of experience in utility regulation. Peyser mounted an effort to gain confirmation, but after the New York Times editorialized against his nomination, he withdrew.

Fortune smiled on Peyser in 1978 when the popular, young Republican Congressman who replaced him, Bruce Caputo, left his seat to run for Lieutenant Governor of New York. Peyser jumped into the Democratic primary and easily defeated a young Assemblyman named Richard Brodsky. Peyser easily gained election in 1978 and again in 1980.

During his four years in Congress as a Democrat, Peyser aligned himself closely with the leadership of Speaker "Tip" O'Neill. He continued his battle for improved student loan programs and fought for labor law Reform.

The 1980 census brought on another round of redistricting in New York and the loss of three congressional seats. The Republican Senate in Albany exacted its revenge on the "turncoat" congressman. They carved his district into three pieces and left him only one realistic option besides retirement, a campaign against his popular friend, Rep. Benjamin Gilman, a Republican. The new 20th district was far from the compact urban/suburban district Peyser had represented. It extended almost 200 miles in length and covered territory from suburban Westchester and Rockland to rural counties like Orange and Sullivan in the Catskills. Only 20f the voters had been in Peyser's congressional district. Gilman won comfortably.

Peyser made an attempt at a comeback in 1984, running in a Democratic primary in an adjacent district in which he did not live. He finished a disappointing third.

Following his political career, Peyser re-entered the financial sector, where he assisted investment management firms in growing their portfolios through his relationships with labor unions and insurance companies. He worked for firms such as Daseke & Co,, Sanford C. Bernstein & Co. and Gabelli Asset Management, where (as of 2005) he still works at the age of 84.