Kinsey was born in Baltimore, Maryland in 1773. He attended the common schools, and in early life engaged in the manufacture of paper. He moved to Bloomfield Township, Essex County, New Jersey and later to Paterson, New Jersey and New Prospect (now Waldwick), Bergen County, N.J., continuing in the paper industry.

Kinsey was a member of New Jersey General Assembly in 1812, 1813, 1819, and 1826 and served in the State council in 1814. He was elected as a Republican to the 15th Congress (March 4, 1817 - March 3, 1819), then elected to the 16th Congress to fill the vacancy caused by the resignation of John Condit and served from February 2, 1820 to March 3, 1821.

After serving in the Congress, Kinsey continued to work in the paper industry. He then became a judge, serving as judge of the court of common pleas and of the orphans’ court of Bergen County, New Jersey from 1830 to 1845. He died in New Prospect, N.J. on June 25, 1849 and is buried in Union Cemetery, near New Prospect.