Conason was born in New York City. He graduated from White Plains High School, in White Plains, New York. In 1971, as a senior, he edited an underground newspaper distributed throughout Westchester County called the Paper Workshop. After attending community college for a year, Conason received a B.A. in history from Brandeis University in 1975. He then worked at two Boston-based newspapers, East Boston Community News and The Real Paper.

From 1978 to 1990, he worked as a columnist and staff writer at The Village Voice.

In The Free Voice of Labor, a 1980 documentary movie about the Yiddish anarchist newspaper Freie Arbeiter Stimme (or Free voice of labor), a young Joe was interviewed. His grandfather Joseph Cohen served as the paper's editor for a number of years and Conason may have been an intern for them.

From 1990 to 1992, Conason was "editor-at-large" for Details magazine. In 1992, he became a columnist for the New York Observer, a position he still holds.

He served as investigative editor for The American Prospect.

In 1992 Conason wrote an article for Spy magazine naming Jennifer Fitzgerald and Jane Morgan as women who allegedly were having affairs with George H. W. Bush, using Linda Tripp as a source.

Conason was a regular guest and a guest host on The Al Franken Show, where he had the distinction of being the only guest with two theme songs. He made an appearances every Friday as a commentator, as well as co-judging with Al their weekly quiz show Wait, wait, don't lie to me.

Conason coined the term "corporate-jet conservative" as a counter to "limousine liberal" (a term used to describe wealthy, insulated supporters of left-wing causes).

In 2000, he co-authored the book The 10 Year Campaign to Destroy Bill and Hillary Clinton with Gene Lyons; the book was turned into a documentary in 2004, which Conason co-produced. The Raw Deal, his book on the Bush Administration's efforts to "end Social Security as we know it," appeared in 2005. In 2007, Conason published Authoritarian Peril in the Age of Bush (Thomas Dunne Books).

Conason briefly appeared in part 2. of the Adam Curtis documentary The Power Of Nightmares talking about the Neoconservatives fantasy enemy in the Clinton-Lewinsky scandal. He also appears in the Stefan Forbes documentary The Lee Atwater Story. In the film he says, “Atwater perceived, long before anyone else in American politics did, that the biggest threat to Bush was a guy from Arkansas named Bill Clinton. The idea was to dirty up Clinton, to do enough damage to him that he would simply be too damaged to run for President. Distract and divert. Atwater started transforming politics into a series of tabloid moments in a way that was incredibly powerful for the Republicans.”

He married Elizabeth Wagley, in 2002. They have two children, boy and girl twins, born in 2007.