Ehrmann was of German descent; both his parents emigrated from Bavaria in the 1840s. Young Ehrmann was educated at the Terre Haute Fourth District School and the German Methodist Church.

He received a degree in English from DePauw University in Greencastle, Indiana, which he attended from 1890 to 1894. While there, he was a member of Delta Tau Delta's Beta Beta chapter and was editor of the school newspaper, Depauw Weekly.

Ehrmann then studied philosophy and law at Harvard University, where he was editor of Delta Tau Delta's national magazine The Rainbow, circa 1896.

Ehrmann returned to his hometown of Terre Haute, Indiana in 1898 to practice law. He was a deputy state's attorney in Vigo County, Indiana for two years. Subsequently, he worked in his family's meatpacking business and in the overalls manufacturing industry. At age 40, Ehrmann left the business to write. At age 54, he wrote Desiderata, which achieved fame only after his death.

Ehrmann was awarded a Doctor of Letters honorary degree from DePauw University in about 1937. He was also elected to the Delta Tau Delta Distinguished Service Chapter, the fraternity's highest alumni award.

Ehrmann died in 1945. He is buried in Highland Lawn Cemetery in Terre Haute, Indiana. In 2010 the city honored Ehrmann with a life-size bronze statue by sculptor Bill Wolfe. He is depicted sitting on a downtown bench, pen in hand, with a notebook in his lap. A portion of "Desiderata" is engraved on a plaque that resides next to the statue.