Kapleau was born in New Haven, Connecticut. As a teenager he worked as a bookkeeper. He briefly studied law and later became an accomplished court reporter. In 1945 he served as chief Allied court reporter for the "Trial of the Major War Criminals Before the International Military Tribunal", which judged the leaders of Nazi Germany. This was the first of the series commonly known as the Nuremberg Trials.

Kapleau later covered the International Military Tribunal for the Far East, commonly known as the Tokyo War Crimes Trials. While in Japan he became intrigued by and drawn to Zen Buddhism. Specifically, he attended a number of informal lectures given by D.T. Suzuki in Kita Kamakura. After returning to America, he renewed his acquaintance with D.T. Suzuki who had left Kita Kamakura to lecture on Zen at Columbia University. But disaffected with a primarily intellectual treatment of Zen, he moved to Japan in 1953 to seek Zen's deeper truth.

He trained initially with Soen Nakagawa (1907—1984), then rigorously with Daiun Harada (1871—1961), at Hosshin-ji. Later he became a disciple of Haku'un Yasutani (1885—1973), himself a dharma heir of Harada. After 13 years' training, Kapleau was ordained by Haku'un Yasutani in 1965 and given permission to teach. The two later ended their relationship over disagreements about teaching and other personal issues. By Kapleau's own admission, he had not completed k?an study and had not gone further than the Blue Cliff Record, about one third of the k?ans in the Yasutani Roshi curriculum. The koan collections that Kapleau did not study include the Book of Serenity (sometimes called the Book of Equanimity), The Transmission of the Lamp, The Five Ranks, and the Precept Koans of which there are more than 100. Kapleau passed the miscellaneous koans (about 50) the Mumonkan (96 koans including the verses) and the Blue Cliff Record (100) if he finished it. Yamada Roshi claims that Kapleau had not gone further than number 37 of the Blue Cliff Record. Kapleau claims otherwise. Kapleau is therefore an independent teacher who was not an official representative of the Yasutani lineage. The Kapleau lineage begins with him. Nonetheless, there is no doubt that Kapleau and his many students' and Dharma heirs' effect on Zen in the west is remarkable.

Kapleau transcribed other Zen teachers' talks, interviewed lay students and monks, and recorded the practical details of Zen Buddhist practice. His book, The Three Pillars of Zen, was published in 1965, has been translated into 12 languages, and is still in print. It was one of the first English-language books to present Zen Buddhism not as philosophy, but as a pragmatic and salutary way of training and living. During a book tour in 1965 he was invited to teach meditation at a gathering in Rochester, New York. In 1966 he left Japan to create the Rochester Zen Center. In doing so, he became the first American to found and teach at a Zen training center.

For almost 40 years, Kapleau taught at the Center and in many other settings around the world, and provided his own dharma transmission to several disciples of both genders. He also introduced many modifications to the Japanese Zen tradition, such as chanting the Heart Sutra in the vernacular English in the U.S., or Polish at the Center he founded in Katowice. He often emphasized that Zen Buddhism adapted so readily to new cultures especially because it was not dependent upon a dogmatic external form. At the same time he recognized that it was not always easy to discern the form from the essence, and one had to be careful not to "throw the baby out with the bathwater."

Throughout the 1970s Toni Packer accepted minor teaching positions at Rochester Zen Center. In 1981 she ran the Center in Kapleau's absence and was in line to be his successor. Packer left the Center shortly after Kapleau's return and ceased practicing Buddhism.

Kapleau was an articulate and passionate writer. His emphasis in writing and teaching was that insight and enlightenment are available to anyone, not just austere and isolated Zen monks. Also well-known for his views on vegetarianism, peace and compassion, he remains widely read, and is a notable influence on Zen Buddhism as it is practiced in the West. Today, his dharma heirs, descendants and former students teach at Zen Centers around the world.

He lived with Parkinson’s Disease for several years, and while his physical mobility was reduced, he enjoyed lively and trenchant interactions with a steady stream of visitors throughout his life. On May 6, 2004, he died peacefully in the backyard of the Rochester Zen Center, surrounded by many of his closest disciples and friends.

A favorite saying of Philip Kapleau was "Grist for the mill" which means that all of our troubles and trials can be useful or contain some profit to us. In the spirit of this his gravestone is one of the mill-stones from Chapin Mill, the 135-acre (0.55 km2) Buddhist Retreat center whose land was donated by a founding member of the Rochester Zen Center, Ralph Chapin.