He served in various municipal offices and one term in the Massachusetts House of Representatives (1902-1903). He is noted for having been elected to the Board of Aldermen in 1904 while in prison, having been convicted of fraud. Curley and an associate, Thomas Curley (no relation) took the civil service exams for postmen for two men in their district to help them get the jobs with the federal government. Though the incident gave him a dark reputation in respectable circles, it aided his image in working class or poor circles because they saw him as a man willing to stick his neck out to help a poor man. He was elected to the U.S. House of Representatives in the 12th district seat(1911-1914). He served three terms as Mayor of Boston (1914-1918, 1922-1926 and 1930-1934), served as Governor of Massachusetts (1935-1937), was returned to the U.S. House of Representatives (1943-1947), this time in the 11th district, and then served one more term as Mayor of Boston from 1945-1949. During his last mayoral term he was convicted for a second time on federal charges of official misconduct. He spent five months in jail during this term, but still retained a considerable degree of popularity with the working classes. The city manager, John Hynes, ran the city during his incarcaration, and intentionally held many large items in limbo until Curley got released from prison so the mayor could handle them himself. Upon release Curley told the manager he was grateful for what he had done, but then told the media that he had accomplished more in his first day back as mayor than the manager had over the previous several months. Livid, Hynes felt betrayed, and this anger fueled Hynes' successful attempt for mayor in 1949. A failed mayoral bid in 1951 marked the end of his serious political career, although he continued to support other candidates and remain active within the Democratic Party, and even ran for mayor one last time in 1955. That was his 10th time running for Boston's mayor.

Mayor Curley was convicted of mail fraud in 1947, but was pardoned by President Harry Truman after serving just five months of his prison term.

Curley had an unusually tragic personal life. He outlived his first wife and seven of his nine children. Two twins died shortly after childbirth. One of his two daughters died while a teenager. His namesake, James Jr., who was groomed as Curley's political heir, died in his early adulthood. Another son, who had a drinking problem, died while Curley ran for mayor in 1945. Finally, his remaining daughter and another son both died of strokes on the same day in 1950. Both were in the same room of Curley's house talking on the same phone when they had their two strokes. Two other sons outlived Curley. One became a Jesuit.