Abercrombie was born in Buffalo, New York. Upon graduating from Williamsville High School (now Williamsville South High School), he went on to pursue studies at Union College in Schenectady, New York. There, Abercrombie obtained his bachelor's degree in 1959 and his master's degree in 1964. He moved to Honolulu, Hawaii to become a public school teacher at various elementary and high schools in the Hawaii State Department of Education. He also pursued work as a probation officer for the Hawaii State Judiciary. While teaching, Abercrombie studied at the University of Hawaii at Manoa where he obtained his doctorate in 1974. With his doctorate, Abercrombie became a sociologist and joined the University of Hawaii System faculty as a professor.

Abercrombie first participated in a political campaign in 1970, having sought the Democratic nomination to run for the United States Senate. Unsuccessful, Abercrombie ran for the Hawaii State House of Representatives where he served from 1975 to 1979. He moved to the upper chamber winning a seat in the Hawaii State Senate where he served from 1979 to 1986. When Cecil Heftel resigned from the United States Congress to run for Governor of Hawaii, Abercrombie entered the race for the Democratic nomination to replace Heftel and won, he was then elected to the US Congress in the following special election. He served from September 20, 1986 to January 3, 1987 but was an unsuccessful candidate for relection in 1986. Abercrombie then set his sights on a seat of the Honolulu City Council. He won the race and served from 1988 to 1990. At the end of his council tenure, Abercrombie once again ran for Congress and won, being reelected in 1992, 1994, 1996, 1998, 2000, 2002 and 2004.