Living in San Jose, Brown studied creative writing at San Francisco State University and then attended the University of California, Irvine where he received an MFA degree in creative writing. His first short story was published when he was 16. His first novel, Going Fast (1977), published in a limited edition by Border Mountain Press, was reviewed by Merritt Clifton in Samisdat:

Going nowhere at 16, bored and frustrated with school, aware of the world's flaws but feeling helpless to correct them, James Brown's hero/narrator, Virgil, idly picks up a spray paint can and says in bold script what he thinks of it all, on the side of an abandoned building. Accosted by two burly policement, in an instant he is Going Fast. First to reform school, where he meets JT, a confirmed incorrigible. Then into a low-paying, grimy job, living in the San Jose slum. Learning from JT, becoming his partner, Virg gradually claims his own destiny and identity, through ten rapidly moving, freewheeling days in and around San Francisco drug traffic... There are no bad or good guys, angels or devils, only people, who can be and often are both.

His second novel, Hot Wire (Arbor House, 1985), focuses on the struggles of a waitress and her three sons. The semi-autobiographical Final Performance (Sceptre, 1988), about two brothers in Los Angeles, was reviewed in Library Journal by Kimberly G. Allen, who commented, "Its characters imbued with an honest emotional depth, this work is compelling and profoundly moving."

He followed with The Second Story Theatre and Two Encores (Story Line Press, 1994), collecting together a novella and two short stories, "The Rat Boy" and "The Friend." His novel Lucky Town (Harcourt, 1994) follows a young boy who runs away from a foster home to meet his ex-con father. When The Los Angeles Diaries was published by HarperCollins in 2003, Publishers Weekly reviewed:

Brown's tales are harrowing: at five, he and his mother traveled from their San Jose home to San Francisco, where she set an apartment building ablaze. Arson couldn't be proven, but she was imprisoned for tax evasion. At nine, he shared his first drink and high with his siblings; when he was 12, a neighbor attempted to molest him; by 30 he was an alcohol- and cocaine-addicted writer-in-residence. During his marriage's early years, Brown often left his wife to feed his addictions, repeatedly promising her he'd reform. Desperate to fuel his writing career, he attempted screenwriting, but everything he pitched seemed too dark. Brown's genius compels readers to sympathize with him in every instance. Juxtaposed with the shimmery unreality of Hollywood, these essays bitterly explore real life, an existence careening between great promise and utter devastation. Brown's revelations have no smugness or self-congratulation; they reek of remorse and desire, passion and futility. Brown flays open his own tortured skin looking for what blood beats beneath and why. The result is a grimly exquisite memoir that reads like a noir novel but grips unrelentingly like the hand of a homeless drunk begging for help.

His personal essays have appeared in GQ, The New York Times Magazine, The Los Angeles Times Magazine and Ploughshares. His writing has also been featured in Denver Quarterly and New England Review. He has been anthologized in Best American Sports Writing of 2006, Fathers and Sons and Sports: An Anthology of Great American Sports Writing (ESPN, 2008).

This River, scheduled for July 2010 publication by Phoenix Books, is a continuation of The Los Angeles Diaries, picking up where the first memoir ended with many of the same themes of family, addiction and recovery. Phoenix is also reissuing the trade paperback of Los Angeles Diaries with a new introduction.

Brown received the Nelson Algren Award for Short Fiction, a National Endowment for the Arts Fellowship in fiction writing and a Chesterfield Film Writing Fellowship from Universal/Amblin Entertainment. He was a Visiting Fellow at the Bread Loaf Writers' Conference,

He teaches in the M.F.A. Program at California State University, San Bernardino and lives with his family in Lake Arrowhead, California.