McIlvanney was born in the town of Kilmarnock, the son of a miner. He studied at Kilmarnock Academy and later at the University of Glasgow, after which he worked as an English teacher between 1960 and 1975. His first book, Remedy is None, was published in 1966 and won the Geoffrey Faber Memorial Prize. Docherty (1975), a moving portrait of a miner whose courage and endurance is tested during the depression, won the Whitbread Novel Award. The Big Man (1985), is the story of Dan Scoular, an unemployed man who turns to bare-knuckle fighting to make a living. Both novels feature typical McIlvanney characters - tough, often violent, men locked in a struggle with their own nature and background.

His novel, The Kiln (1996), is the story of Tam Docherty, the grandson of the hero of Docherty. It won the Saltire Society Scottish Book of the Year Award. The Big Man (1985) was made into a film starring Liam Neeson and featuring Billy Connolly. Laidlaw (1977), The Papers of Tony Veitch (1983) and Strange Loyalties (1991) are crime novels featuring Inspector Jack Laidlaw. Laidlaw is considered to be the first book of Tartan Noir, despite the author calling the genre " ersatz".

William McIlvanney is also an acclaimed poet, and is the author of The Longships in Harbour: Poems (1970) and Surviving the Shipwreck (1991), which also contains pieces of journalism, including an essay about T. S. Eliot. His short story "Dreaming" (published in Walking Wounded in 1989) was filmed by BBC Scotland in 1990 and won a BAFTA. His brother is the sports journalist Hugh McIlvanney.

He was the narrator of the BBC Scotland football documentary Only a Game? in 1986, and the official history of Celtic football club in 1988.