He is perhaps most famous as the man who almost, but not quite, invented the phonograph. No one before M. Charles Cros had thought of reproducing sound by making an apparatus capable of registering and reproducing sounds which had been engraved with a diaphragm. The inventor gave the name of Paleophone (voix du passé) to his invention. On April 30, 1877 he submitted a sealed envelope containing a letter to the Academy of Sciences in Paris explaining his proposed method. The letter was read in public on the 3rd December following. In his letter, after having shown that his method consisted of detecting an oscillation of a membrane and using the tracing to reproduce the oscillation with respect to its duration and intensity. Cros added that a cylindrical form for the receiving apparatus seemed to him to be the most practical, as it allowed for the graphic inscription of the vibrations by means of a very fine-threaded screw. An article on the Paleophone was published in "la semaine du Clergé" on October 10, 1877, written by l'Abbé Leblanc. Cros proposed metal for both engraving tool attached to the diaphragm and receiving material for durability.

Before Cros had a chance to follow up on this idea or attempt to construct a working model, Thomas Alva Edison introduced his first working phonograph in the USA. Edison used a cylinder covered in tinfoil for his first phonograph, patenting this method for reproducing sound on January 15, 1878. Edison and Cros apparently did not know of each other's work in advance.

Cros was convinced that pinpoints of light observed on Mars and Venus, probably high clouds illuminated by the sun, were the lights of large cities on those planets. He spent years petitioning the French government to build a giant mirror that could be used to communicate with the Martians and Venusians by burning giant lines on the deserts of those planets. He was never convinced that the Martians were not a proven fact, nor that the mirror he wanted was technically impossible to build.

In the early 1870s Cros had published with Mallarmé, Villiers and Verlaine in the short-lived weekly Renaissance littéraire et artistique, edited by Emile Blémont.

His poem The Kippered Herring inspired Ernest Coquelin to create what he called monologues, short theatrical pieces whose format was copied by numerous imitators.The piece, translated as The Salt Herring, was translated and illustrated by Edward Gorey.

L'Académie Charles Cros, the French equivalent of the US Recording Academy, is named in his honor.

Cros was a member of the group known as the hydropathes which existed around the period 1878-1881.Charles Cros, played by Christopher Chaplin, appears in the film Total Eclipse, about the lives of Paul Verlaine and Arthur Rimbaud. Cros is seen for a few seconds at the Le Chat Noir in Paris, a café which opened in 1881 and had become the home for the avant-garde art scene of the time.