McPhee was born in Princeton, New Jersey, the son of the Princeton University athletic department's physician, Dr. Harry McPhee. John was educated at Princeton High School, then spent a postgraduate year at Deerfield Academy, before attending Princeton University and the University of Cambridge.

While at Princeton, McPhee went to New York once or twice a week to appear as the juvenile panelist on the radio and television quiz program Twenty Questions.

Twice married, McPhee is the father of four daughters, among them the novelists Jenny McPhee and Martha McPhee, and photographer Laura McPhee.

McPhee's writing career began at Time magazine and led to a long association with The New Yorker weekly magazine beginning in 1965 and continuing to the present. Many of his twenty-nine books include material originally written for that magazine.

McPhee has received many literary honors, including the Award in Literature from the American Academy of Arts and Letters, and the Pulitzer Prize for General Non-Fiction in 1999, awarded for Annals of the Former World. In 1978 McPhee received a Litt.D. from Bates College, and in 2009 he received an honorary Doctorate of Letters from Yale University.

McPhee's subjects, reflecting his personal interests, are highly eclectic. He has written pieces on lifting body development (The Deltoid Pumpkin Seed), the United States Merchant Marine (Looking for a Ship), farmers' markets (Giving Good Weight), the shifting flow of the Mississippi River (The Control of Nature), geology (in several books), as well as a short book entirely on the subject of oranges. One of his most widely read books, Coming into the Country, is about the Alaska wilderness. His most recent book, Uncommon Carriers, published 16 May 2006, is about freight transportation.

McPhee has profiled a number of famous people, including conservationist David Brower and the young Bill Bradley, whom McPhee followed closely during Bradley's four-year basketball career at Princeton University. The resulting book, A Sense of Where You Are, is a classic of non-fiction writing – a literary craftsman's admiring profile of a basketball craftsman. But some of McPhee's most memorable work describes people who work out of the limelight: a builder of birch bark canoes (Henri Vaillancourt), a bush pilot, and a French-speaking wine maker in the Swiss army.

McPhee is also a renowned nonfiction writing instructor at Princeton University, having taught generations of aspiring undergraduate writers, many of whom have achieved distinction in literature and journalism. Among his former students are David Remnick, Pulitzer Prize-winning author and current editor of The New Yorker; Richard Stengel and Jim Kelly, the current and former managing editors of Time magazine; journalist Robert Wright, former senior editor at The New Republic and columnist for Time, Slate and the New York Times, and author of award-winning books; and Peter Hessler, The New Yorker's China correspondent. McPhee still teaches his writing seminar two years out of every three, most recently during the spring 2009 semester.