Swinnerton-Dyer is the son of Sir Leonard Schroeder Swinnerton Dyer, 15th Baronet, and his wife Barbara, daughter of Hereward Brackenbury. He was a Fellow of Trinity College, Master of St Catharine's College and vice-chancellor of the University of Cambridge from 1979 to 1983. From 1983 he was Chairman of the University Grants Committee and then from 1989, Chief Executive of the Universities Funding Council. He was elected Fellow of the Royal Society in 1967 and was a KBE in 1987. In 2006 he was awarded the Sylvester Medal.

Swinnerton-Dyer was, in his younger days, an international bridge player, representing the British team twice in the European Open teams championship. In 1953 at Helsinki he was partnered by Dimmie Fleming (the only occasion a woman has played in the British Open team): the team came second out of fifteen teams. In 1962 he was partnered by Ken Barbour; the team came fourth out of twelve teams at Beirut.

Swinnerton-Dyer has a Morphy Number of 3.