Harvey was born in Glasgow. His musical roots were in Dixieland jazz and skiffle music, which enjoyed considerable popularity in England and Scotland during the late 1950s. During this period, he won a competition that sought "Scotland's answer to Tommy Steele". Alex Harvey was literally the "last of the teenage idols," a distinction he made much of during his subsequent career - practically worshipped by his fans.

In 1959, Harvey formed "Alex Harvey's Soul Band," and recorded blues and rock and roll material, to modest success. In 1966, Harvey found more success as a member of the cast in London production of the musical Hair.

In 1972, Harvey formed the Sensational Alex Harvey Band, composed of himself, guitarist Zal Cleminson, bassist Chris Glen, and cousins Ted and Hugh McKenna on drums and keyboards respectively. All four were former members of a progressive rock band called Tear Gas. The Sensational Alex Harvey Band (or SAHB, as they were often known) gave Harvey his greatest successes, both musical and sales. Harvey's skiffle background led him to include unusual material for the period in his sets, including old pop songs and Broadway musical material. His live act also usually featured a tale of "Vambo", an urban superhero who was the subject of some of his more energetic numbers. His performances were more substantial than the usual rock musicians of the time combining a musical and verbal flare with both humour and sincerity and his songs often contained messages and morals. The concept albums "The Impossible Dream" and "Tomorrow Belongs To Me" rival even the Beatles' and The Rolling Stones' late 60s albums for richness of texture.

The Sensational Alex Harvey Band scored chart hits in Britain with the single Delilah, a remake of the Tom Jones hit, and also with "The Boston Tea Party".

Harvey died of a heart attack in Zeebrugge, Belgium, the day before his 47th birthday.