Smith was born in Reading, Pennsylvania. He was educated at the University of Pennsylvania, and received a Bachelor of Arts in Creative Writing in 1964. From 1965 to 1969 he worked as a journalist, and began writing fiction in the early 1970s. His first mystery, featuring a Gypsy art dealer in New York named Roman Grey, Canto for a Gypsy (1973), was nominated for an Edgar Award. Nightwing, also an Edgar nominee, was his breakthrough novel, and he adapted it for a 1979 feature film.
Smith is best known for his novels featuring Russian investigator Arkady Renko. In 1981, Smith wrote Gorky Park, which was called the "thriller of the '80s" by Time Magazine. It became a bestseller and won a Gold Dagger Award from the British Crime Writers' Association. Renko has since appeared in six other novels by Smith.
In the 1970s, Smith wrote two Slocum adult action westerns under the pen name Jake Logan. Smith also wrote a number of other paperback originals, including a series about a character named 'The Inquisitor', a James Bond-type agent employed by the Vatican.
He originally wrote under the name "Martin Smith", only to discover there were other writers with the same name. His agent asked Smith to add a third name and Smith chose Cruz, his paternal grandmother's surname.