He was born in Derry on the 13 April 1824, the third child of Rev Robert Alexander. He was educated at Tonbridge School and Brasenose College, Oxford.

After holding several livings in Ireland he was made bishop of Derry and Raphoe in 1867. In 1896 he became Archbishop of Armagh and Primate of All Ireland. He was the last bishop to sit in the House of Lords before the Disestablishment of the Church of Ireland in 1871.

He gave the Bampton Lectures in 1876. An eloquent preacher and the author of numerous theological works, including Primary Convictions, he is best known as a master of dignified and animated verse. His poems were collected in 1887 under the title of St Augustine's Holiday and other Poems.

His wife, Cecil Frances Alexander, wrote some tracts in connection with the Oxford Movement. She is known as the author of hymns such as Once in Royal David's City, All Things Bright and Beutiful and many other well known hymns. They both lived in Milltown House, Strabane. The house is now used as a school, Strabane Grammar School.

Bishop Alexander is mentioned as part of the procession in James Joyce's cyclops episode of Ulysses.