Gurganus was born in Rocky Mount, North Carolina. He first trained as a painter, studying at the University of Pennsylvania and the Pennsylvania Academy of Fine Arts. He served three years with the United States Navy during the Vietnam War and began writing during his time on the USS Yorktown. He graduated from Sarah Lawrence College where he studied with Grace Paley. He studied with John Cheever and Stanley Elkin at the University of Iowa in the Iowa Writers' Workshop. Cheever sold Gurganus's short story "Minor Heroism" to The New Yorker without telling Gurganus beforehand.

In addition to later teaching at both Sarah Lawrence and the Iowa Writers' Workshop, he has also taught at Stanford and Duke Universities.

His best known work is his 1989 debut novel, Oldest Living Confederate Widow Tells All, which was on the New York Times Best Seller list for eight months. It won the Sue Kaufman Prize from The American Academy of Arts and Letters, was a main selection of the Book-of-the-Month Club, and sold over four million copies. It was made into a CBS television play, with Cicely Tyson winning one of its four Emmy Awards as best supporting actress in the role of the freed slave Castalia. The novel was also adapted for a one-woman Broadway play, starring Ellen Burstyn, in 2003.

Gurganus's other works include White People, a collection of short stories and novellas; Plays Well With Others, a novel; and The Practical Heart, a collection of four novellas, which won a 2001 Lambda Literary Award in the Gay Men's Fiction category. His shorter fiction has been published in The New Yorker, The Atlantic Monthly, and The Paris Review, in addition to being included in the O. Henry Prize Collection and the Norton Anthology of Short Fiction.

After living in New York City for a number of years, Gurganus returned to North Carolina, where he co-founded the political group Writers Against Jesse Helms and, as a result, appeared as himself in Tim Kirkman's 1998 documentary Dear Jesse. Gurganus has also taken a position against the Iraq War, most notably by citing his Vietnam War experience in an essay published in The New York Times Magazine, "The War at Home", published April 6, 2003, a few weeks after the invasion. Gurganus was also the inaugural guest editor of New Stories From the South, an annual collection of notable fiction by Southern writers published by Algonquin Books of Chapel Hill, in 2006.

He was the recipient of a 2006 Guggenheim fellowship.