Rogers was born at Newington Green, then a village north of Islington, London. His father, Thomas Rogers, a banker, was the son of a Stourbridge glass manufacturer, who was also a merchant in Cheapside. Thomas married Mary, the only daughter of his father's partner, Daniel Radford, becoming himself a partner shortly afterwards. On his mother's side Samuel Rogers was connected with the well-known Nonconformist clergymen Philip Henry and his son Matthew, was brought up in Nonconformist circles, and became a long-standing member of the Unitarian congregation at Newington Green, then led by the remarkable Dr Richard Price. He was educated at private schools at Hackney and Stoke Newington.

He wished to enter the Presbyterian ministry, but his father persuaded him to join the banking business in Cornhill. In long holidays, necessitated by delicate health, Rogers became interested in English literature, particularly the work of Samuel Johnson, Thomas Gray and Oliver Goldsmith. He learned Gray's poems by heart, and his family wealth allowed him to leisure to try writing poetry himself. He began with contributions to the Gentleman's Magazine, and in 1786 he published a volume containing some imitations of Goldsmith and an "Ode to Superstition" in the style of Gray.

In 1788 his elder brother Thomas died, and Samuel's business responsibilities were increased. In the next year he paid a visit to Scotland, where he met Adam Smith, Henry Mackenzie, Hester Thrale and others. In 1791 he was in Paris, and enjoyed the Orleans Collection of art at the Palais Royal, many of the treasures of which were later to pass into his possession. With Gray as his model, Rogers took great pains in polishing his verses, and six years elapsed after the publication of his first volume before he printed his elaborate poem on The Pleasures of Memory (1792) — regarded by some as the last embodiment of the poetic diction of the 18th century. The theory of elevating and refining familiar themes by abstract treatment and lofty imagery is taken to extremes. In this art of "raising a subject", as the 18th century phrase was, the Pleasures of Memory is much more perfect than Thomas Campbell's Pleasures of Hope, published a few years later in imitation. Byron said of it, "There is not a vulgar line in the poem."

In 1793 his father's death gave Rogers the principal share in the banking house in Cornhill, and a considerable income. He left Newington Green and established himself in chambers in the Temple. In his circle of friends at this time were Richard Sharp (Conversation Sharp) and the artists John Flaxman, John Opie, Martin Shee and John Henry Fuseli. He also made the acquaintance of Charles James Fox, with whom he visited the galleries in Paris in 1802, and whose friendship introduced him to Holland House. In 1803 he moved to 22 St James's Place, where for fifty years he entertained all the celebrities of London. Flaxman and Charles Alfred Stothard had a share in the decoration of the house, which Rogers virtually rebuilt, and proceeded to fill with works of art. His collections at his death realized £50,000.

An invitation to one of Rogers's breakfasts was a formal entry into literary society, and his dinners were even more select. His social success was due less to his literary position than to his powers as a conversationalist, his educated taste in all matters of art, and no doubt to his sarcastic and bitter wit, for which he excused himself by saying that he had such a small voice that no one listened if he said pleasant things. "He certainly had the kindest heart and unkindest tongue of any one I ever knew," said Fanny Kemble. He helped the poet Robert Bloomfield, he reconciled Thomas Moore with Francis Jeffrey Jeffrey and with Byron, and he relieved Sheridan's difficulties in the last days of his life. Moore, who refused help from all his friends, and would only owe debts to his publishers, found it possible to accept help from Rogers. He procured a pension for HF Cary, the translator of Dante, and obtained Wordsworth his sinecure as distributor of stamps.

Rogers was in effect a literary dictator in England. He made his reputation by The Pleasures of Memory when William Cowper's fame was still in the making. He became the friend of Wordsworth, Walter Scott and Byron, and lived long enough to give an opinion as to the fitness of Alfred Tennyson for the post of Poet Laureate. Alexander Dyce, from the time of his first introduction to Rogers, was in the habit of writing down the anecdotes with which his conversation abounded. In 1856 he arranged and published selections as Recollections of the Table-Talk of Samuel Rogers, to which is added Porsoniana. Rogers himself kept a notebook in which he entered impressions of the conversation of many of his distinguished friends...Fox, Edmund Burke, Henry Grattan, Richard Porson, John Horne Tooke, Talleyrand, Lord Erskine, Scott, Lord Grenville and the Duke of Wellington. They were published by his nephew William Sharpe in 1859 as Recollections by Samuel Rogers; Reminiscences and Table-Talk of Samuel Rogers, Banker, Poet, and Patron of the Arts, 1763—1855 (1903), by GH Powell, is an amalgamation of these two authorities. Rogers held various honorary positions: he was one of the trustees of the National Gallery; and he served on a commission to inquire into the management of the British Museum, and on another for the rebuilding of the Houses of Parliament.

He was elected a Fellow of the Royal Society in November 1796.

His literary production remained slow. An Epistle to a Friend (the above-mentioned Conversation Sharp), published in 1798, describes Rogers's ideal of a happy life. This was followed by The Voyage of Columbus (1810), and by Jacqueline (1814), a narrative poem, written in the four-accent measure of the newer writers, and published in the same volume with Byron's Lara. His reflective poem on Human Life (1819), on which he had been engaged for twelve years, is written in his earlier manner.

In 1814 Rogers made a tour on the Continent with his sister Sarah. He travelled through Switzerland to Italy, keeping a full diary of events and impressions, and had made his way to Naples when the news of Napoleon's escape from Elba obliged him to hurry home. Seven years later he returned to Italy, paying a visit to Byron and Shelley at Pisa. Out of the earlier of these tours arose his last and longest work, Italy. The first part was published anonymously in 1822; the second, with his name attached, in 1828. It was at first a failure, but Rogers was determined to make it a success. He enlarged and revised the poem, and commissioned illustrations from J.M.W. Turner, Thomas Stothard and Samuel Prout. These were engraved on steel in the sumptuous edition of 1830. The book then proved a great success, and Rogers followed it up with an equally sumptuous edition of his Poems (1834). In 1850, on Wordsworth's death, Rogers was asked to succeed him as poet laureate, but declined the honour on account of his age. For the last five years of his life he was confined to his chair in consequence of a fall in the street. He died in London, and is buried in the family tomb in the churchyard of St Mary's Church, Hornsey High Street, Haringey .