Killens was born in Macon, Georgia, to Charles Myles, Sr., and Willie Lee Killens. His father Charles encouraged him to read Langston Hughes's writings and his mother Willie Lee, president of Dunbar Literary Club, introduced him to poetry. He was an enthusiastic reader as a child and was inspired by writers such as Hughes and Richard Wright. His great-grandmother’s tales of slavery were another important factor in his gaining knowledge of traditional black mythology and folklore, which later appeared in his writings. John Oliver Killens planned to be a lawyer.

Killens attended a number of institutions including Morris Brown College, Howard University, and Columbia University but never earned a degree. In 1939, Killens attended Robert Teller Law School for a few years before he joined the army during World War II in which he became a member of the Pacific Amphibian Forces. He abandoned the idea of becoming a lawyer and concentrated on writing instead.

In 1948, Killens moved to New York and focused on establishing a literary career. He attended writing classes at Columbia University and New York University. He was an active member of many organizations such as the National Labor Relations Board (NLRB) and the Congress of Industrial Organizations (CIO). Around 1950, Killens co-founded a writer’s group that became the Harlem Writers Guild.

His first published novel associated with the Harlem Writers Guild was Youngblood (1954), dealing with a fictional black Georgia family in the early 1900s. His second novel was And Then We Heard the Thunder (1962) about the treatment of the black soldiers in the military. His third novel, Sippi (1967) focused on the voting rights during the 1960s. Two of Killen’s novels earned Pulitzer Prize nominations, And Then We Heard the Thunder (1962) and The Cotillion or One Good Bull Is Half the Herd[1] (1971). In 1986, Killens founded the National Black Writers Conference at Medgar Evers College where he taught English. In 1987, Killens died of cancer in Brooklyn, NY.