Koeppen was born out of wedlock in Greifswald to a seamstress. His father never accepted the fatherhood formally. He lived first in his grandmother's house on Bahnhofstrasse, but moved after her death in 1908 together with his mother to her sister in Ortelsburg (Szczytno), East Prussia, where Koeppen started visiting the public school. He and his mother moved back to Greifswald in 1912, but only two years later returned to East Prussia. Koeppen came back to Greifswald after World War I, working as a delivery boy for a book dealer. During that time he volunteered at the theater and attended lectures at the University of Greifswald. Finally in 1920, Koeppen left Greifswald permanently, and after twenty years of moving about, he settled in Munich, living there the remainder of his life.

He started as a journalist. In 1934 his first novel appeared while he was in the Netherlands. In 1939 he came back to Germany, and from 1943 lived in Munich, where he also died in 1996 at age 89.

In 1947, Koeppen received a book contract to rewrite the memoirs of the philatelist and Holocaust survivor Jakob Littner (born 1883 in Budapest, died 1950 in New York City). The resulting novel caused some controversy based on whether Koeppen was given a written manuscript to guide his work on Littner, and the novel never sold well. In 1992, a new edition was published, which led to the discovery of Littner's original text. In 2000, Littner's original manuscript was published in English and in 2002, in German.

Koeppen's last major novel Der Tod in Rom (Death in Rome) was published in 1954. In the ensuing years, Koeppen found it difficult to complete longer works. In 1951, Koeppen had published his novel Tauben im Gras (Piegeons on the Grass), which utilized a stream of consciousness literary technique and is considered a significant work of German-language literature by Germany's foremost literary critic Marcel Reich-Ranicki. Throughout the 1950s, Koeppen travelled extensively, to the U.S., the Soviet Union, London and Warsaw. Between 1962 and 1987, he received numerous literary prizes in the Federal Republic of Germany.