Holland was born in a cottage in the grounds of the ancient Sheffield Manor in Yorkshire and initially trained by his father to follow him as a maker of optical instruments. He was however a bookish young man who taught himself Latin and soon began publishing poems of his own. These eventually brought him to the notice of local poet James Montgomery, editor of the Sheffield Iris, who published both articles and poems of his in the paper, although commenting on the latter's 'inaccuracies and imperfections' and remarking that 'they would be twice as good if they were as short again'. In 1813 Holland had become a Sunday School teacher and turned his hand to composing religious poetry and hymns. Some five years later he was appointed as a secretary of the Sheffield Sunday School Union.

In 1819 Holland wrote a long topographical poem on Sheffield Park and sent it to Montgomery for corrections and suggestions. Previously his output had largely been anonymous or appeared under his initials; the poem now became the first book published under his name and was followed by another in 1821 on the 'plague village' of Eyam. That year too he published the narrative poem "The Cottage of Pella" in imitation of Montgomery's "The Wanderer of Switzerland". Another strong influence on him at this time was the poet Thomas Campbell. Holland's poem "The Rainbow" (1820), published at the same time as one on the same subject by Campbell, was frequently anthologised as the latter's. Following this, Holland took Campbell's The Pleasures of Hope as the model for his own The Hopes of Matrimony (1822). As well as subsequent productions, a good deal of poetry was also included in some of his prose works such as the serialised The Old Arm Chair (1823), his botanical work Memoirs of the Rose (1824) and his consideration of the crucifiction, Cruciana (1836). Since he continued to write at length on all occasions, readers have to mine such gems as are there from among much that is derivative and diffuse.

In 1825, Holland was appointed as editor of the Sheffield Iris newspaper by new owner John Blackwell. In 1832, he moved to Northumberland to edit the Newcastle Courant, which Blackwell had bought, but returned to Sheffield in the new year and was elected curator of the Sheffield Literary and Philosophical Society, a position he held until his death. In 1835 he became co-editor of the Sheffield Mercury, serving until the paper was closed down by its new owner in 1848. Throughout his journalistic life, he wrote numerous works on botany, geology, local history and topography, in addition to biographies and sermons.

The Methodist (anonymously), Liverpool, 1819;Sheffield Park: a descriptive poem, Sheffield, 1820; annotated 2nd ed. 1859;The Cottage of Pella: a tale of Palestine, Sheffield, 1821;The Village of Eyam: a poem, Macclesfield, 1821;The Hopes of Matrimony, London and Sheffield, 1822; 2nd ed. 1836;Flowers from Sheffield Park: a selection of poetical pieces originally published in the Sheffield Iris, London and Sheffield, 1827;The Pleasures of Sight: a poem, Sheffield, 1829;Tyne Banks: a poetical sketch by a visitor in Newcastle, Newcastle, 1832;A Poet’s Gratulation, Sheffield, 1851;Diurnal Sonnets: 366 poetical meditations on various subjects, personal, abstract and local, comprising several founded on the more striking festivals and observances of the Christian year, Sheffield, 1852.

:::From Sheffield Park, stanzas LVII-LVIX

:::When winter evening's cheerful tales conspire

:::With the warm influence of a social fire,

:::How seldom thinks the happy midnight guest

:::Of the poor collier's brief and broken rest.

:::Where chemic nature, from sulfureous ores,

:::Her deadliest essence sublimates and stores -

:::Combines these dire arcana to prepare

:::Her noxious treasures of mephitic air,

:::Each moment hovering round the miner's lamp

:::To scorch or suffocate - the explosive damp;

:::Above his head, while threatening rocks impend,

:::Imprisoned spirits in their wombs contend:

:::He delves his dungeon vault of living coal

:::And hears the cataracts through the caverns roll,

:::Careless with every stroke, or every breath,

:::To rouse a danger or inhale a death.

:::'Tis his to know, 'midst all that pity craves,

:::The felon's task, the heritage of slaves,

:::'Doomed to the mines', to dig for others' wealth,

:::To earn subsistence, and to bury health -

:::Bear from earth's noisesome depths, with perils rife,

:::The curse, the comforts, or the bread of life.

:::This a sad proof how vainly man hath built

:::Pride's superstructure on a base of guilt;

:::Of penal judgment this the unvarying mark,

:::From far Potosi's mines to Sheffield Park.