Koch (pronounced coke) was born Jay Kenneth Koch in Cincinnati, Ohio. He began writing poetry at an early age, discovering the work of Shelley and Keats in his teenage years. At the age of 18, he served in WWII as a U.S. Army infantryman in the Philippines.

After his service, he attended Harvard University, where he met future New York School poet John Ashbery. After graduating from Harvard in 1948, and moving to New York City, Koch studied for and received his Ph.D. from Columbia University.

In 1951 he met his first wife, Janice Elwood, at UC Berkeley; they married in 1954 and lived in France and Italy for over a year. Their daughter, Katherine, was born in Rome in 1955. In 1959 he joined the faculty in the Department of English and Comparative Literature at Columbia, and he taught classes at Columbia for over forty years.

His first wife died in 1981; Koch married his second wife, Karen Culler, in 1994. He was inducted into the American Academy of Arts and Letters in 1996. Koch died from a year-long battle with leukemia in 2002.

While a student at Harvard, Koch won the prestigious Glascock Prize in 1948. In 1962, Koch was writer in residence at the New York City Writer's Conference at Wagner College.

The 1960s saw his first published books of poetry, but his poetry did not garner wider popular acclaim until the 1970s with his book The Art of Love: Poems (1975). He continued writing poetry and releasing books of poetry up until his death. Koch won the Bollingen Prize for One Train (1994) and On The Great Atlantic Rainway: Selected Poems 1950-1988 (1994), followed closely by the Phi Beta Kappa Poetry Award winner New Addresses (2000).

In 1970, Koch released a pioneering book in poetry education, Wishes, Lies and Dreams: Teaching Children To Write Poetry. Over the next 30 years, he followed this book with other books and anthologies on poetry education tailored to teaching poetry appreciation and composition to children, adults, and the elderly.

Koch wrote hundreds of avant-garde plays over the course of his 50 year career, highlighted by drama collections like 1000 Avant-Garde Plays (1988), which only contains 116 plays, many of them only one scene or a few minutes in length. His prose work is highlighted by The Red Robins (1975), a sprawling novel about a group of fighter pilots flying for personal freedom under the leadership of Santa Claus. He also published a book of short stories, Hotel Lambosa (1988), loosely based on and inspired by his world travels. He also produced at least one libretto, and several of his poems have been set to music by composers.

Koch taught poetry at Columbia University, where his classes were popular. His wild humor and intense teaching style, often punctuated by unusual physicality (standing on a table to shout lines by Walt Whitman) and outbursts of vocal performance often drawn from Italian opera, drew non-English majors and alumni. Some of the spirit of these lectures is contained in his final book on poetry education, Making Your Own Days (1998). His students included poets Ron Padgett, David Shapiro, Alan Feldman, David Lehman, Jordan Davis, Jessy Randall, David Baratier, Loren Goodman, and Carson Cistulli.

His poems were translated in German by the poet Nicolas Born in 1973 for the renowned "red-frame-series" of the Rowohlt Verlag.

Koch had a brush with the infamous anti-art affinity group Up Against the Wall Motherfuckers in early January 1968. During a poetry reading at St. Mark's Church, a member of the group walked in and pointed a handgun at the podium, shouting "Koch!" before firing one blank round. The poet regained his composure and said to the "shooter," "Grow up."

Koch asked in his poem Fresh Air (1956) why poets were writing about dull subjects with dull forms. Modern poetry was solemn, boring, and uneventful. Koch described poems “Written by the men with their eyes on the myth/ And the missus and the midterms” He attacked the idea that poetry should be in any way stale.

Koch wrote of how:

    The Waste Land gave the time’s most accurate data,
    It seemed, and Eliot was the Great Dictator
    Of literature.  One hardly dared to wink
    Or fool around in any way in poems,
    And critics poured out awful jereboams
    To irony, ambiguity, and tension —
    And other things I do not wish to mention.
                   (Excerpt from ‘''''Seasons on Earth'''',’ 1987)

Though not against T. S. Eliot, Koch opposed the idea that in order to write poetry one had to be depressed or think that the world is a terrible place. His ideas were developed with close friends Frank O'Hara and John Ashbery, along with painters Jane Freilicher and Larry Rivers, among others. He once remarked that “Maybe you can almost characterize the poetry of the New York School as having as one of its main subjects the fullness and richness of life and the richness of possibility and excitement and happiness.” In his poem, The Art of Poetry (1975), Koch offered guidelines to writing good poetry. Among his 10 suggestions are “1) Is it astonishing?” and “10) Would I be happy to go to Heaven with this pinned on to my angelic jacket as an entrance show? Oh would I?”

Koch once remarked that “Children have a natural talent for writing poetry and anyone who teaches them should know that.” In his poems:

1. He mixed word usage with various levels of imagery;
2. He set two contrasting tones next to each other, simplicity and silliness at the same time;
3. He spoke to everything, animate and inanimate objects;
4. He used parody of other poets to express his own views, both serious and comic.

Koch was labeled by some as just a comedic poet. He acknowledged this in an interview and offered his comments:

He gives a picture of this in “To Kidding Around,” where the joys of being a joker are proclaimed:

    To be rid of troubles
    Of one person by turning into
    Someone else, moving and jolting
    As if nothing mattered but today
    In fact nothing
    But this precise moment...
              (Excerpt from To Kidding Around, 2000)

Koch collaborated with the composer Ned Rorem on an opera, Bertha, which received its premier in 1973. His short play, George Washington Crossing the Delaware, was produced in 1962. Numerous others of his plays have been produced.

• Days and Nights (1982)

• From the Air (1979)

• Ko: or, A Season on Earth (1959)

• On the Edge (1986)

• On the Great Atlantic Rainway: Selected Poems 1950—1988 (1994)

• One Train (1994)

• Permanently (1961)

• Poems (1953)

• Bertha, & other plays (1966)

• Poems from 1952 and 1953 (1968)

• Seasons on Earth (1987)

• Sleeping with Women (1969)

• Straits (1998)

• Thank You and Other Poems (1962)

• The Art of Love (1975)

• The Burning Mystery of Anna in 1951 (1979)

• The Duplications (1977)

• The Pleasures of Peace and Other Poems (1969)

• When the Sun Tries to Go On (1969)