He was born 24 February 1763 at Mildred's Court London. He became a glazier, but on the death of his father he used his inheritance to study for the Unitarian ministry, where he became proficient in mathematics and Latin. Later he became tutor to the sons of Earl Stanhope.

Joyce had radical political views, became a member of the Society for Constitutional Reform and of the London Corresponding Society. On 4 May 1794 he was arrested at Stanhope's house in Kent on a charge of 'treasonable practices', and examined by the Privy Council, where he refused to answer any questions. He remained in custody until 19 May when, with others, he was committed to the Tower of London on a charge of High Treason. They were arraigned at the Old Bailey on 25 October. After the acquittal of some of his co-defendants, the prosecution withdrew the case against him and he was released. Joyce had suffered twenty three weeks imprisonment, and his employer and friends worked ceaselessly to ensure his acquittal.

Joyce was for many years the secretary of the Unitarian Society. At the time of his death, 21 June 1816, he was minister of the Unitarian chapel at Hampstead. He left a widow and six children.

Joyce wrote a number of popular educational works on science and mathematics; he was largely responsible for the editing of George Gregory's Dictionary of Arts and Sciences (1808) and William Nicholson's British Encyclopaedia (1809). He also contributed articles to Rees's Cyclopaedia (1802-1819).