Kalstone, born in McKeesport, Pennsylvania, was the recipient of a Fulbright scholarship and studied at the University of Cambridge. He taught at Harvard University starting in 1959 and was a professor of English at Rutgers University from 1967 until his death. He was a closeted homosexual.

He lectured on and wrote about 20th century poets including Elizabeth Bishop and Robert Lowell. His friends included the poet James Merrill and the writer Edmund White , who is said to have portrayed Kalstone as the character Joshua in the novel The Farewell Symphony .