He attended the public schools of Lowell, Massachusetts and then Harvard University from 1859 to 1862; he quit school and returned home after his father died. He taught school in Chelmsford, Massachusetts and studied law. During the Civil War he served with the Union Army in New Bern, North Carolina for five months. He was admitted to the bar in Lowell in 1865 and served in the common council of Lowell in 1868 and 1869. He then became a member of the school committee in 1871, holding that post until 1873. He had a private legal practice and was also a judge in the Lowell Police Court from 1874 to 1884.

On October 1, 1872, Greenhalge married Isabella (or Isabel) Nesmith, and they had three children: Nesmith Greenhalge (1873-1874), Frederic B. Greenhalge (1875-?), and Harriet Nesmith Greenhalge (1878-?).

He was then mayor of Lowell in 1880 and 1881 and an unsuccessful candidate for election to the Massachusetts Senate in 1881. He was a delegate to the Republican National Convention in 1884 and a member of the Massachusetts House of Representatives in 1885 but was unsuccessful in his bid for reelection. He became city solicitor in 1888, practicing law in Middlesex and other counties. He was elected as a Republican to the Fifty-first Congress, serving from March 4, 1889 to March 3, 1891 but failed in 1890 to be re-elected to Congress. However, he was subsequently elected Governor of Massachusetts after a ferocious 1893 campaign and served from January 1894 until his death in Lowell. While governor, the Commonwealth paid off its last public debt and he proclaimed the first Patriots' Day, ending the 200-year-old Fast Day celebration in Massachusetts. Perhaps his greatest test in office came in February 1894 when an angry mob of 5,000 gathered in front of the State House to demand unemployment subsidies; he came out of his office to address them and promise them relief, whereupon their anger subsided. When he died, businesses and schools closed in his honor. At his funeral Senator Henry Cabot Lodge and Harvard President Charles Elliot served as pallbearers; he is buried in Lowell Cemetery.

Preceded by William Russell Governor of Massachusetts (1894-1896) Succeeded by Roger Wolcott Governors of Massachusetts  Colony  Endicott • Winthrop • T. Dudley • Haynes • Vane • Winthrop • T. Dudley • Bellingham • Winthrop • Endicott • T. Dudley • Winthrop • Endicott • T. Dudley • Endicott • Bellingham • Endicott • Bellingham • Leverett • Bradstreet

 Dominion  J. Dudley • Andros • Bradstreet

 Province  W. Phips • Stoughton • Coote • Stoughton • Governor's Council • J. Dudley • Tailer • Shute • Dummer • Burnet • Dummer • Tailer • Belcher • Shirley • S. Phips • Shirley • S. Phips • Pownall • Hutchinson • Bernard • Hutchinson • Gage