Lucie-Smith was born in Kingston, Jamaica, moving to the United Kingdom in 1946. He studied at The King's School, Canterbury, and after a little time in Paris he also studied at Merton College, Oxford.

After serving in the Royal Air Force and working as a copywriter, he became a full-time writer. He succeeded Philip Hobsbaum in organising The Group, a London-centred poets' group.

In an academic programme about pornography, he appeared naked while being painted by a female artist. The purpose was to show how rarely this happens compared to art made by men where women are naked and men are clothed. At the beginning of the 1980s he conducted several series of interviews, Conversations with Artists, for BBC Radio 3.