Nishiyama was born on October 10, 1928, in Tokyo, Japan. His father was a lawyer and a kendo master, and he had two sisters, Sumiye and Michiko. During Nishiyama's childhood, karate was mostly unknown in Japan, and children were required to train in either judo or kendo in middle school. He started learning kendo in 1933 and judo in 1938. Recalling the beginning of his martial arts training, Nishiyama said, "I began, as is the custom in Japan, on the 5th day of the fifth month at the age of five. I started in kendo." His kendo instructor, Moorio Mochida, had a strong and lasting influence on the youth's development in the martial arts. By the time Nishiyama reached 14 years of age, he had attained black belt status in judo.

In 1943, Nishiyama was promoted to the rank of 1st dan in kendo (and would eventually reach 3rd dan in that art), and began training in Shotokan karate at the art's honbu dojo (headquarters training hall) under its founder, Gichin Funakoshi. His interest in karate had been sparked by two things: an old book written by a sword master (Nishiyama estimated the book to have been written around 1550), and Zoku Sugata Sanshiro (1945), a film by Akira Kurosawa. He recalled that training in the honbu dojo was around 80% kata (patterns) and 20% kihon (basics), and that Funakoshi and his son, Gig? Funakoshi, would divide the teaching load evenly between them.

Nishiyama served in the Imperial Japanese Navy before the end of World War II, and was unable to continue his karate training during this brief period. In 1946, he was promoted to the rank of 1st dan in karate, and in 1948, he was promoted to 2nd dan. With the war over, he was able to begin studying at university.

While enrolled at Takushoku University, Nishiyama became a member of the university's karate team, and in 1949 he was named team captain. That same year, he helped to establish the Japan Karate Association (JKA). He also co-founded the All Japan Collegiate Karate Union, and was elected its first chairman. In 1950, Funakoshi promoted him to 3rd dan in karate.

Nishiyama received a Master of Arts degree in economics from Takushoku University in 1951. That year, he was appointed to the JKA's Board of Directors. He began working for Shell Oil, but his JKA role soon became his full-time occupation. He was appointed to take charge of the JKA instructors' training program, and continued to oversee it until the late 1950s. Recalling notable graduates of the course, he said, "There were many but I think Mr. Kanazawa was exceptional, even though he left the JKA. There were many very good people, like Mr. Enoeda, Shirai, Mikami and many others as well."

In 1952, Nishiyama began teaching karate to US military personnel from the Strategic Air Command (SAC). The other karate instructors for this program included Masatoshi Nakayama and Isao Obata. The following year, United States Air Force General Curtis LeMay invited several instructors, including Nishiyama, to tour US air bases on the North American mainland. In 1960, Nishiyama published his first book, Karate: The Art of Empty-Hand Fighting (co-authored with Richard Brown). This work has been recognized as a seminal textbook on karate; the cover of a 1990 reprint noted that over 300,000 copies had been sold. In June 1960, Nishiyama was promoted to the rank of 5th dan in Shotokan karate.

In July 1961, Nishiyama moved to the United States of America at the invitation of his students from the SAC, and founded the All American Karate Federation (AAKF). It was around this time that Tsutomu Ohshima invited Nishiyama to take charge of his karate students in the US, as Ohshima was planning to return to Japan following completion of his US university studies. The arrangement did not work out satisfactorily, and was to be a source of bitterness between the two men. Four years later, in 1965, Nishiyama organized the first United States vs. Japan Goodwill Karate Tournament. He wrote a long letter on the state of karate tournaments, published in 1967 by Black Belt magazine. In that letter, he warned of the dangers of unqualified competitors and officials. In 1968, he organized the first World Invitational Karate Tournament, held at the Los Angeles Memorial Sports Arena.

In 1973, Nishiyama co-founded the Pan American Karate Union and was elected its first Executive Director. He became Executive Director of the International Amateur Karate Federation (IAKF) in 1974. The first IAKF World Karate Championship was held in Los Angeles in 1975. The organization would later become the International Traditional Karate Federation (ITKF) in 1985. In 1979, the AAKF became the American Amateur Karate Federation (and kept the same acronym). Nishiyama was elected as the inaugural President of the new AAKF.

Through the 1980s, Nishiyama served in a variety of administrative leadership roles, including President of JKA International (USA), Chairman of the AAKF, and President of the ITKF. He was based in Los Angeles, where he had settled with his family, but travelled across the world to teach karate. Nishiyama was featured on the cover of the December 1986 issue of Black Belt magazine. In 1989, he published the Traditional Karate Coach's Manual.

In the early 1990s, Nishiyama's refusal to align his ITKF organization with the World Union of Karate-Do Organizations (WUKO) caused the International Olympic Committee to suspend its recognition of WUKO as amateur karate's international governing body. The ITKF and WUKO had been due to merge and form a unified karate organization, but this did not eventuate. The WUKO eventually became the World Union of Karate-Do Federations in late 2008.

In 1999, the United States Senate honored Nishiyama by presenting him with the US flag that had been flown on October 10 (his birthday) that year over the US Capitol in Washington, D.C. The flag was awarded in recognition of his contributions to traditional karate. At the request of US Senator Slade Gorton, the accompanying letter stated: "I would like to thank you for your many years of service to this country. Your efforts in bringing the art of Karate to the United States Armed Forces has been immeasurable. On behalf of your grateful students and friends, I wish you the best in your many endeavors."

In May 2000, Nishiyama was honored when the Nishiyama Cup was held in Moscow, the first official karate event conducted in Russia since the end of the Soviet Union. On November 3, 2000, the Emperor of Japan awarded the Order of the Sacred Treasure, 4th Class (Gold Rays with Rosette), to him in a ceremony on the grounds of the Tokyo Imperial Palace. Nishiyama was one of only 11 recipients of the award from outside Japan in that year.

During the first Traditional Karate World Cup, in October 2001, Poland honored Nishiyama with the Officer's Cross of the Order of Merit of the Republic of Poland. The award was presented by the President of Poland, Aleksander Kwa?niewski. On November 1, 2003, the International San Ten Karate Association (ISKA) awarded Nishiyama the rank of 10th dan. The ISKA reported that "Master Nishiyama at first tried to turn aside the honor, but was moved by the heartfelt affection of the ISKA leadership." The calligraphy for the certificate was executed by Eri Takase.

Nishiyama died on November 7, 2008, in Los Angeles, following a long battle with lung cancer. His funeral was held on November 30 at the Hompa Hongwanji Buddhist Temple in Los Angeles, with over 500 people in attendance. He was survived by his wife, Yohko, and their three daughters: Yumi, Amy, and Nami.

On October 10, 2009, the ITKF posthumously awarded the rank of 10th dan to Nishiyama; his daughter Nami accepted the honor on his behalf. ITKF Chairman Richard Jorgensen recalled, "[Nishiyama] always said he never wanted to achieve the level of 10th Dan because he believed to do so and acquire the highest rank meant that his learning and his life would be over ... he would never accept the rank while he lived."