He was born in Kupin, Ukraine, to a Jewish family under the name Ivan Greenberg. He made his way to the United States by way of Palestine and worked as a teacher of Hebrew.

He joined the American Communist Party in 1932. He is noted for his role in founding Partisan Review with William Phillips in 1933. The journal broke with the Soviet line in 1937 in the wake of the Moscow Trials and maintained an ongoing feud with Stalinist Popular Front advocates such as Granville Hicks of New Masses. As an independent publication, Partisan Review went on to become the most influential literary journal of the period.

Philip Rahv was a beacon of the New York intelligentsia. When the narrator of Robert Lowell's poem, Man and Wife meets his future wife, he "outdrank the Rahvs in the heat/of Greenwich Village, fainting at your feet." Rahv's work at Partisan Review (which he co-founded) put him at the center of an intellectual circle that included Dwight Macdonald, Lionel Trilling, Hannah Arendt, Mary McCarthy, Alfred Kazin, Delmore Schwartz, Sidney Hook, and many other intellectuals of the period. Rahv remained a Marxist and was committed to the idea of achieving a synthesis of radical social criticism and literary excellence.

He is also known for his later hostility toward myth-criticism in the style of Northrop Frye. As he put it, "What the craze for myth represents most of all is the fear of history."

Rahv taught at Brandeis University in his later years and died in Cambridge, Massachusetts, in 1973 in what appeared to be suicide.