Potter, the third son of John Potter (fl. 1676-1723), a prebendary of Wells Cathedral, studied at Emmanuel College, Cambridge and graduated BA in 1742, when was ordained. He married the daughter of Rev. Colman of Hardingham. His children included a daughter Sarah, referred to in a letter. He became curate of Reymerston and vicar of Melton Parva, but the combined emoluments of these were less than £50 a year. He later became curate of Scarning, Norfolk, as well as master of Seckar's School there from 1761 to 1789, but spent much of his time writing and translating. Among his pupils was Jacob Mountain (1749-1825), the first Anglican bishop of Quebec.

In 1788, Potter was appointed by the bishop of Norwich vicar of the combined parishes of Lowestoft and Kessingland, Suffolk, and thanks to the patronage of Lord Thurlow, the lord chancellor, who had attended Seckar's School a prebendary of Norwich Cathedral. According to one story, Thurlow and Potter had been schoolfellows at Seckar's, which seems unlikely as Seckar was ten years his junior. For whatever reason, when Potter approached Thurlow to ask for a £10 subscription to his Sophocles translation, he received the valuable cathedral stall instead.

Robert Potter died aged 83 and was buried in Lowestoft churchyard. There is a 1789 etching of a bewigged Potter in the National Portrait Gallery in London.

Potter completed versions of the plays of Æschylus (1779), Euripides (1781-3) and Sophocles (1788) that remained in print through the 19th century. Of the three, the Æschylus was best known, as there were no other translations of the author available before the 1820s. His scheme of using blank verse for Greek hexameters and rhymed verse for the choruses was widely adopted by other translators. He also published copious poetry, some sermons, and some political pamphlets,, targeted, for instance, at the "pretended inspiration of the Methodists" (1758) and at the Poor Laws (1775).

It emerges from a letter from Sarah Burney to her sister Frances Burney on August 1, 1779 that Samuel Johnson, Hester Thrale and their circle had a poor opinion of Potter's poetic abilities. Potter, on the other hand, developed reservations about Johnson's literary judgement, which he expanded on a few years later. Johnson may have described Potter's poetry as "verbiage", but Horace Walpole was welcoming: "There is a Mr. Potter too, I don't know who, that has published a translation of Aeschylus, and as far as I have looked is a good poet."