Barclay was born at Gordonstoun in Moray, Scotland. His father Col. David Barclay of Urie had served under Gustavus Adolphus, and pursued a somewhat tortuous course through the troubles of the civil war. His mother was Katherine Gordon (1620—1663) the daughter of Sir Robert Gordon 1st Bart of Gordonstoun (1580—1654). He was the eldest of five children.

Robert was sent to finish his education at the Scots College, Paris, of which his uncle was Rector, and made such progress in study as to gain the admiration of his teachers, specially of his uncle, who offered to make him his heir if he would remain in France, and join the Roman Catholic Church.

In 1667, however, he followed the example of his father, and joined the recently formed Society of Friends after returning to Scotland. Soon afterwards he began to write in defence of the movement, by publishing in 1670 Truth cleared of Calumnies, and a Catechism and Confession of Faith (1673). In 1670 he had married another Quaker, Christian Mollison (c.1651—1724), of Aberdeen. They had seven children: three sons and four daughters.

He was an ardent theological student, a man of warm feelings and considerable mental powers, and he soon came prominently forward as the leading apologist of the new doctrine, winning his spurs in a controversy with one William Mitchell. The publication of fifteen Theses Theologiae (1676) led to a public discussion in Aberdeen, each side claiming a victory. The most prominent of the Theses was that bearing on immediate revelation, in which the superiority of the Inward Light of Christ to reason or scripture is sharply stated. He was noted as a strong supporter of George Fox in the controversies that tore into Quakers in the 1670s. His greatest work, An Apology for the True Christian Divinity, was published in Latin at Amsterdam in 1676, and was an elaborate statement of the grounds for holding certain fundamental positions laid down in the Theses. It was translated by its author into English in 1678, and is claimed to be "one of the most impressive theological writings of the century" . It is a weighty and learned work, written in a dignified style, and was eagerly read.

It, however, failed to arrest the persecution to which the Quakers were exposed, and Barclay himself, on returning from Europe, where he travelled extensively (once with William Penn and George Fox), and had several interviews with Elisabeth, Princess Palatine, was several times thrown into prison, but soon regained his liberty, and was in the enjoyment of Court favour.

In later years he had much influence with James II, who as Duke of York had given New Jersey to Sir George Carteret and John Berkeley, 1st Baron Berkeley of Stratton. After Carteret's death his half (East Jersey) was sold in 1682 to twelve people, eleven of whom were members of the Society of Friends. One of the eleven Quaker proprietors was William Penn, and after expanding to include a larger number of proprietors, the group elected Barclay to be the governor. He is said to have visited James with a view to making terms of accommodation with William of Orange, whose arrival was then imminent.

His latter years were spent at his estate of Ury, where he died.

The essential view which Barclay maintained was that all people can be illuminated by the Inward Light of Christ which is the author of the Scriptures and will lead them into all truth. His works have often been reprinted.