He was born in Dublin on 22 July 1846, the son of The Rt. Rev Charles Graves, bishop of Limerick, by his wife Selina, the daughter of John Cheyne (1777—1836), the Physician-General to the Forces in Ireland. Alfred was educated in England at Windermere College, and Trinity College, Dublin. His paternal grandmother Helena was a Perceval, and the granddaughter of the Earl of Egmont. His grandfather, John Crosbie Graves, was a first cousin of 'Ireland's most celebrated surgeon', Robert James Graves.

In 1869 he entered the Civil Service as clerk in the Home Office, where he remained until he became an inspector of schools in 1874 . He was a contributor of prose and verse to the Spectator, The Athenaeum, John Bull, and Punch magazine.

He took a leading part in the revival of Irish letters. He was for several years president of the Irish Literary Society, and was the author of the famous ballad of Father O'Flynn and many other songs and ballads. In collaboration with Charles Stanford he published Songs of Old Ireland (1882), Irish Songs and Ballads (1893), the airs of which are taken from the Petrie MSS.; the airs of his Irish Folk-Songs (1897) were arranged by Charles Wood, with whom he also collaborated on Songs of Erin (1901).

He published an autobiography, To Return to All That in 1930, as a response to his son Robert's Goodbye to All That.