Anderson was born near Philadelphia, Pennsylvania. He served in the American Revolutionary War and afterward was admitted to the Delaware bar. He was appointed United States judge of the Southwest Territory in 1791. In 1796 he served as a delegate to the convention that drafted the first Tennessee state constitution. In 1797 he was elected by the Tennessee General Assembly to fill the vacancy in the Senate created by that body's expulsion of the seat's original occupant, William Blount. That term was scheduled to expire on March 3, 1799; however, on December 12, 1798 the Tennessee General Assembly elected Anderson to the state's other Senate seat, the term for which was to expire on March 3, 1803.

Anderson was to serve in this seat continuously until his retirement from the Senate effective March 3, 1815. He was reelected by the General Assembly in 1803 and 1809. In the Eighth Congress (1803-1804), he served as President Pro Tempore of the United States Senate. After retiring from the Senate, he remained in Washington, D.C. and he was appointed Comptroller of the U.S. Treasury and served in that office from 1815 until 1836, dying the next year. He was interred in the Congressional Cemetery in the Anacostia section of Washington. Anderson County, Tennessee is named in honor of Joseph Anderson. His son Alexander Outlaw Anderson later served as a Senator from Tennessee.

Many of the biographical details contained in this article are derived from the public domain Biographical Directory of the United States Congress.