Klein graduated from the Hackley School and the University of Pennsylvania with a degree in American civilization. In 1969, Klein began reporting for the Essex County Newspapers, and The Peabody Times in Massachusetts In 1972, he reported for Boston's WGBH, and until 1974 he was also the news editor for The Real Paper in Cambridge, Massachusetts. He was a contributing editor for Rolling Stone from 1975 to 1980, and Washington bureau chief from 1975 to 1977. He became friends with actor-director Tom Laughlin after interviewing him for Rolling Stone and appeared briefly as a reporter in Laughlin's 1977 film Billy Jack Goes to Washington.

Klein published Woody Guthrie: A Life in 1980 and Payback: Five Marines After Vietnam in 1984. He was a political columnist for New York from 1987 to 1992 where he won the Peter Kihss Award for reporting on the 1989 race for Mayor of New York. In May 1992 he joined Newsweek and wrote the column "Public Lives", which won a National Headliner Award in 1994. Newsweek also won a National Magazine Award for their coverage of Bill Clinton's 1992 victory. From 1992 to 1996 he was also a consultant for CBS News, providing commentary.

In January 1996, Klein anonymously published the novel A Novel of Politics, based on the 1992 Democratic presidential primary. The book spent nine weeks as number one on the New York Times bestseller list, with its author listed as "Anonymous". Several people, including former Clinton speechwriter David Kusnet and, later, Vassar professor Donald Foster correctly identified Klein as the novel's author, based on a literary analysis of the book and Klein's previous writing. Klein lied, denying authoring the book and publicly condemning Foster. Klein denied authorship again in Newsweek, speculating that another writer wrote it. Washington Post Style editor David von Drehle, in an interview, asked Klein if he was willing to stake his journalistic credibility on his denial, to which Klein agreed. On July 17, 1996, Klein admitted that the speculation had been correct.

In December 1996, he joined The New Yorker to write the "Letter from Washington" column. In 2000 he published The Running Mate, a sequel of sorts to Primary Colors. In March 2002 Klein published The Natural: Bill Clinton's Misunderstood Presidency, an account of Clinton's two terms in office.

In January 2003, he joined Time to write a column called "In the Arena" on national and international affairs. It appears in Time's upfront "Notebook" section and has been criticized for its reporting about Speaker of the House Nancy Pelosi and the Democratic opposition to warrantless wiretapping. The column has been the source of several retractions by Time.

Klein is a regular blogger on time.com's Swampland blog. In November 2007, Salon columnist Glenn Greenwald wrote about factual errors in a Klein story about the Foreign Intelligence Surveillance Act (FISA). Klein reported that the Democratic version of the FISA bill "would require the surveillance of every foreign-terrorist target's calls to be approved by the FISA court" and that it therefore "would give terrorists the same legal protections as Americans." Glenn Greenwald - Political Blogs and Opinions - Salon Time later published a comment: "In the original version of this story, Joe Klein wrote that the House Democratic version of the Foreign Intelligence Surveillance Act (FISA) would allow a court review of individual foreign surveillance targets. Republicans believe the bill can be interpreted that way, but Democrats don't." Greenwald noted that the text of the legislation does not require court review of individual targets, and that Time's response disregards this fact. Klein's response was, "I have neither the time nor legal background to figure out who's right."

Later, Greenwald reported that Time "refused the requests of two sitting members of Congress ... to correct Klein's false statements in Time itself". Greenwald has reported that Senator Russ Feingold has been informed by Time that his letter rebutting Klein will be published in a forthcoming issue. Glenn Greenwald - Political Blogs and Opinions - Salon

In The Natural, his book about the Clinton presidency, Klein gave a mixed assessment of Clinton's time in office. In the book, he wrote: "the conventions of journalism prevent me from fitting too neatly into one political niche (although as a columnist for the New Yorker and Newsweek my predilections are obvious)". Klein's depiction of the Clinton presidency also gave a detailed examination of the moderate Democratic positions espoused by the Democratic Leadership Council, as well as third way politics generally, of which Klein was highly complimentary.

He is an admirer of George W. Bush personally, although he has sometimes disagreed with his policies. In an interview with Hugh Hewitt Klein said of Bush, "Let me say that of all the major politicians I've covered in presidential politics in the last two or three times around, he is the most likely to stick with an issue, even if the polls are bad, and to govern from the gut as you said. I don't always agree with the decisions that he makes, but I think he is an honorable man, and when I've criticized him, I've tried to criticize him on the substance, and certainly not on his personality, because I really like the guy."

In 2008, Klein caused controversy with comments on the motivations of neoconservatives, when he said:

American foreign policy scholar Max Boot and the Anti-Defamation League National Director, Abraham Foxman, were among the critics of Klein's views. Klein is also frequently criticized by Bob Somerby, a media commentator.

In May 2009 he invited further controversy when he was quoted in an article in Politico.com, wherein he stated that the reasoning and ideas of prominent conservative commentator Charles Krauthammer were of limited value because of Krauthammer's wheelchair use:

"There's something tragic about him... His work would have a lot more nuance if he were able to see the situations he's writing about."

Klein has been criticized by several conservative publications for accusing Fox News host Glenn Beck, Republican Senator Tom Coburn and former Alaskan Governor Sarah Palin of sedition.

Joe Klein wrote the following in Time Magazine, September 13, 2010, In the Arena, "Tough Issues. What if we gave people real choices and real consequences and let them make decisions?" In this article, Klein writes the following: "By most accounts it has succeeded brilliantly, even though the participants are not very sophisticated: 60% are farmers."

Klein lives with his wife, daughter Sophie and son Teddy, in Pelham, New York, and is also the father of two adult sons.[4]

In March 2008, Klein aroused controversy after making what many saw as an anti-Catholic remark in regards to NBC's Tim Russert, Chris Matthews and the New York Times' Maureen Dowd related to their coverage of Hillary Clinton. He later apologized. Others, such as media critic Bob Somerby, defended Klein's comment as accurate.