He was the son of Sir George Sitwell, 4th Baronet of Renishaw Hall. His mother was the former Lady Ida Emily Augusta Denison, a daughter of the Earl of Londesborough and a granddaughter of Henry Somerset, 7th Duke of Beaufort. She claimed a descent through female lines from the Plantagenets. In his later life he became more reticent about associating himself with the publicity attaching to the Sitwells collectively, travelling and concentrating on writing. He became the 6th baronet, inheriting the title when Osbert died in 1969. He was made a Companion of Honour in 1984. His main residence was Weston Hall, Northamptonshire, the family home.

The youngest child of Sir George Reresby Sitwell, he was born in Scarborough, Yorkshire and brought up in Derbyshire; he was educated at Eton College and Balliol College, Oxford. In World War I he served from 1916 in the British Army, in the Grenadier Guards.

After the war he went to Balliol but did not complete a degree; and was heavily involved in Osbert and Edith's projects. In 1925 he married a Canadian, Georgia Doble. Constant Lambert set to music The Rio Grande, one of his poems, and it was performed and broadcast in 1928.

Because his poetry was so severely criticised by those who disliked the Sitwells in general, and although Canons of Giant Art is a work of very considerable impact, he refused to publish any of his poems for many years. In 1967 Derek Parker published a selection of his poems in the summer edition of Poetry Review, including his elegy for his beloved sister Edith. Among his most remarkable and original works are a series of lengthly auto-biographical and art-based "fantasias" such as "For Want of the Golden City", "The Hunters and the Hunted", "Dance of the Quick and the Dead"(1936) that defy easy classification. The list of works below is incomplete.