Bartlett was born in Plymouth, Massachusetts, to William Bartlett and Susan Thacher. His father was a descendant of Mayflower Pilgrims Love Brewster, a founder of the town of Bridgewater, Massachusetts; Elder William Brewster, the Pilgrim colonist leader and spiritual elder of the Plymouth Colony; and John Alden, who was hired as a cooper in Southampton, England, just before the voyage to America and who along with the Brewsters were signers of the Mayflower Compact. His mother was a direct descendant of Anthony and Elizabeth Thacher who were the sole survivors of a terrible shipwreck on August 14, 1635 in which twenty one passengers including their four children were drowned. Thacher Island, a small island off Cape Ann on the Massachusetts coast in the United States is named after them.

A very bright boy, he was reading at age three and had read the entire Bible by nine. He finished school at age sixteen and went to Cambridge, Massachusetts, where he worked for the University Bookstore that served Harvard. By age twenty-nine he owned the store. Known for his memory for quotations and trivia, "Ask John Bartlett" became a byword in the community when someone was stumped.

He began keeping a commonplace book of quotations to answer queries and in 1855 privately printed the first edition of his Familiar Quotations. That edition of 258 pages contained entries from 169 authors. One-third of the book was quotations from the Bible and from the works of William Shakespeare, most of the balance being lines from the great English poets.

Bartlett sold the bookstore in 1862 to become a paymaster in the United States Navy during the Civil War. He served on the South Atlantic station, returning to Boston in 1863 to join the firm Little, Brown and Company. That same year, Little, Brown issued the fourth edition of his quotation book. He rose to be the firm's senior partner in 1878 and retired from the firm in 1889. In addition to work on quotations (he oversaw nine editions of his book), he wrote on fishing, and chess, and compiled a massive concordance of Shakespeare, published in 1894, that is still the standard work of its kind.

The concordance, which Bartlett estimated consumed 16,000 hours of his time, was compiled with his wife Hannah, the daughter of Sidney Willard, a professor of Hebrew at Harvard, and the granddaughter of Joseph Willard, president of Harvard. He died in Cambridge, Massachusetts aged 85 and is buried at Mount Auburn Cemetery in Cambridge, Middlesex County, Massachusetts.

John Bartlett should not be confused with John Russell Bartlett the author of Dictionary of Americanisms.

He was a fellow of the American Academy of Arts and Sciences and received an honorary degree from Harvard in 1871.

His Familiar Quotations is now in its seventeenth edition and is still published by Little, Brown.