Swados's parents, Aaron (a doctor) and Rebecca Bluestone (a pianist, singer, and painter), had Russian Jewish origins. In 1936 he enrolled at the University of Michigan, where he won a Hopwood Award for creative writing. As a high school student and an undergraduate, he had been a member of the Young Communist League, converting to Trotskyism in the late 1930s. He worked as an aircraft riveter after graduating and served in the Merchant Marine between 1943 and 1946 as a seaman and radio operator. He then struggled to support himself as an author for several years, publishing his first novel, Out Went the Candle, in 1955, followed by On the Line in 1957. The latter novel was one of the first literary expressions of US worker discontent during the 1950s. An essay published by Esquire in 1959, "Why Resign from the Human Race?", contributed to the foundation of the Peace Corps. He was named a Guggenheim Foundation Fellow in 1960.
He also taught at Sarah Lawrence and the University of Massachusetts and reported on the Biafran War.