Born near Belfast, Ireland, Graham emigrated to the United States as a child and grew up on a farm in Illinois. There he received an extensive education in business before moving to California in 1851 to search for gold.

In 1860 he settled in Nevada. In partnership with J. W. Mackay, J. C. Flood, and William S. O’Brien, known as the Bonanza Kings, he made a large fortune from the silver of the Comstock Lode. Though he made most of his fortune through mining in California and Nevada, he was also involved in real estate and showed interest in manufacturing.

He was elected as a Democrat to the U.S. Senate in 1881. His wife divorced him in 1883 on grounds of "habitual adultery". He only served one term due to his defeat in the 1886 election. Following the end of his term, he moved back to California, where he died in San Francisco in 1894.

His will left $40 millino in trust to his daughters, Mrs Hermann Oelrichs, and Virginia (later Mrs William Kissam Vanderbilt II) and a son Charles. After his death, Mrs Nettie Cravens came forward claiming to be his wife. She brought plenty of evidence to the court trial, but lost the case. She moved to Iowa and lived in obscurity, spending her last days in a mental institution.

Later, another woman, Phoebe Couzins, a women's-rights advocate also claimed a relationship with Fair.