Young was born in County Tyrone, Ireland, but as a young child his family emigrated to Philadelphia. He entered the newspaper business as a proofreader at age 15. As a reporter for the Philadelphia Press, he distinguished himself with his coverage of the Battle of Bull Run. By 1862 he was managing editor of the Press and another newspaper.

In 1865 he moved to New York. He began writing for Horace Greeley's New York Tribune and became managing editor of that paper. He also began working for the government, undertaking missions to Europe for the US Department of State and the US Department of the Treasury. In 1872, he joined the New York Herald and reported for them from Europe.

Young was invited to accompany President Ulysses S. Grant on Grant's famous 1877-1879 world tour, chronicled in Young's book Around the World with General Grant. Young impressed Grant, especially in China where Young struck up a friendship with Li Hongzhang. Grant persuaded President Chester A. Arthur to appoint Young minister to China in 1882. In this position he distinguished himself by mediating and settling disputes between the US and China and France and China.

In 1885 he resumed working for the Herald in Europe. In 1890 he returned to Philadelphia. In 1897 President William McKinley appointed him Librarian of Congress, the first Librarian confirmed by Congress. During his tenure, the Library began moving from its original home in the US Capitol Building to its own structure, an accomplishment largely the responsibility of his predecessor, Ainsworth Rand Spofford. Spofford served as Chief Assistant Librarian under Young. Young held the post of Librarian until his death.

His brother was Congressman James Rankin Young.