McCarthy joined the Boston Reds in the Union Association in 1884 as a starting pitcher and outfielder. In limited innings and at-bats, he played poorly, losing in all seven of his pitching appearances and batting and paltry .215 when at the plate. McCarthy moved to the National League and played with the Boston Beaneaters the following year and the Philadelphia Quakers the following two years but failed to bat higher than .200 in any season, although in limited at-bats.

Setting aside aspirations of being a star pitcher, McCarthy finally settled into an everyday position in a lineup in 1888 with the St. Louis Browns in the American Association. With the Browns until 1891, McCarthy scored over 100 runs each season and grew increasingly productive at the plate. He batted .350 in 1890 and drove in 95 runs in 1891. Although the shoddy record-keeping of the time prevents an accurate tally from being known, he also asserted himself as a daring presence on the base-paths, by some accounts stealing over 100 bases in 1888 and approaching the mark in 1890.

McCarthy moved back to the National League to play for the Boston Beaneaters in 1892 and enjoyed his most productive seasons over the next few years. In 1893 he drove in over 100 runs for the first time in his career, a feat which he repeated in 1894 while hitting 13 home runs. McCarthy played for the Brooklyn Bridegrooms in 1896 before retiring. He finished his career with a .292 batting average, 44 home runs and roughly 500 stolen bases.

McCarthy was inducted into the Baseball Hall of Fame in 1946.