He was born as John Roy Anderson in the town of Accrington, Lancashire, England, his parents being Albert and Kathleen Anderson. He was later to drop the "h" from his first name, as he had a dream where he was given the name "Jonathan". He attended St. John's Infants School in Accrington, and made a tentative start to his musical career at an early age by playing the washboard in "Little John's Skiffle Group", which played songs by Lonnie Donegan among others.

Anderson left school at the age of fifteen, and went through a series of jobs including working as a farm hand, a lorry driver, and a milkman.

In 1962, Anderson joined The Warriors (also known as The Electric Warriors), where he and his brother Tony shared the role of lead vocalist. He quit this band in 1967, released two solo singles in 1968 under the pseudonym Hans Christian Anderson, and then briefly sang for the bands The Gun and The Open Mind.

In the summer of 1968, Anderson met bassist Chris Squire and joined him a group called Mabel Greer's Toyshop, which had previously included guitarist Peter Banks. Anderson fronted this band, but ended up leaving again before the summer was over. He remarks on his website that his time with the band consisted of "too many drugs, not enough fun!".

Anderson, Squire, and Banks went on to form Yes, with drummer Bill Bruford and keyboardist Tony Kaye. Their debut album was released in 1969. He stayed with the group until 1979. He rejoined a reformed Yes in 1983, and departed again in late 1988 over the band's continued pursuit of major commercial success and mainstream radio play. In 1989, Anderson and other former Yes members formed the group Anderson, Bruford, Wakeman, Howe (ABWH), augmented by bassist Tony Levin who had played with drummer Bill Bruford in King Crimson. After the successful first ABWH album, a bizarre series of business deals caused ABWH to reunite with the then-current members of Yes, who had been out of the public eye while searching for a new lead singer. The resulting eight-man band assumed the name Yes, and the album "Union" was assembled from scraps of an in-progress second ABWH album as well as recordings that "Yes proper" had been working on without Anderson. A spectacular tour followed, but the eight-man lineup of Yes never recorded a complete album together before splintering in 1992. Many more personnel changes followed, but Anderson has been with the band ever since. He appears on all Yes albums except their 1980 album Drama.

Anderson's voice is often described as angelic. He considers himself an alto tenor vocalist, capable of hitting very high notes without relying on falsetto while maintaining the softness that is consistently associated with his voice. Even during live performances, such as those recorded and included in the Yessongs album, Anderson never wavers into falsetto, as many other singers frequently do in live performances to conserve their vocal strength.

Anderson is also responsible for most of the mystically-themed lyrics and concepts which are part of many Yes releases. These elements are crucial components of the classic Yes sound, but have occasionally alienated some members of the band (most notably Bruford and Rick Wakeman), causing them to leave the group. The lyrics are frequently inspired by various books Anderson has enjoyed, from Tolstoy's "War And Peace" to footnotes of the Shastic Scriptures.

Nicknamed "Napoleon" by his bandmates for his diminutive stature and leadership of Yes, Anderson was fond of experimenting within the band, also adding to the conflict. He originally wanted to record the album Tales From Topographic Oceans in the middle of the woods, and instead decided to put hay and animal cut-outs all over the recording studio, causing lice to infest one of Rick Wakeman's keyboards. In another incident, Anderson had tiles installed in the studio, to simulate the echo effect of one's vocals in a bathroom.

In 1999 Anderson's vocal was featured on the song "The Only Thing I Need" by the Contemporary Christian music band 4Him.

Anderson was a smoker in the 1960s and 70s, preferring Benson & Hedges cigarettes. Before live performances he often meditates in a tent with crystals and dreamcatchers, a practice he started in the 1980s. Anderson is also a vegetarian, as were most members of Yes during the mid-seventies.

Jon Anderson's children include daughter Deborah Anderson (who has done work singing for the French Electronica band, Telepopmusik on the album "Angel Milk" released in summer 2005), Jade Anderson (who has released a solo album in Japan) and Damion Anderson (another musician).