Carroll was born in Washington, D.C., to Thomas Edmund and Marea Grace Carroll on September 27, 1969. He attended Columbia University in New York City, receiving his bachelor's degree in history in 1993. In May 1992, while a junior at Columbia, Carroll was inspired by a lecture by the Joseph Brodsky (the Nobel Prize-winning Poet Laureate of the United States) to found the American Poetry and Literacy Project (APLP). Meeting in a Greenwich Village café in late 1992, Carroll and Brodsky decided that the APLP would distribute poetry books for free to members of the public. Carroll, APLP's executive director, persuaded the Book-of-the-Month Club to donate thousands of copies of poetry books to the APLP. The books were distributed in hotels, hospitals, and homeless shelters and aboard airlines. By 1994, more than 12,500 poetry books had been distributed. Another 15,000 books were given away in 1997. Carroll went on a nationwide tour sponsored by the Academy of American Poets in 1998, promoting the APLP and distributing 100,000 free poetry books at truck stops, hospital waiting rooms, train stations, and jury rooms in courthouses. Volkswagen later paid APLP to put 40,000 poetry books in the glove boxes of its cars in April 1999, and Target Corporation paid APLP for 300,000 books to give away to their customers. Another 100,000 copies of poetry books were distributed at the 2002 Winter Olympics in Salt Lake City. In early 1998, he edited the poetry anthology 101 Great American Poems.

During this period of his life, Carroll also wrote two books about volunteering. He co-wrote a 1991 book (Volunteer USA) about ways to become a volunteer, and followed it up in 1994 with a book on volunteering opportunities for senior citizens.

On November 11, 1998, Carroll founded the Legacy Project, a national, all-volunteer project to seek and preserve wartime correspondence. Carroll was inspired to create the Legacy Project after his family home in Washington, D.C., burned down in 1992 and destroyed most of his and his family's personal correspondence. Carroll then wrote to Abigail Van Buren, author of the popular "Dear Abby" advice column, and asked her to write a column requesting that people preserve any letters they had received from loved ones who were away at war and send them to the Legacy Project. Carroll's letter was published in the "Dear Abby" column on Veterans Day in 1998. Within a year, Carroll had received more than 15,000 letters...some of them originals, and some dating as far back as the American Revolution. News about the Legacy Project caught the attention of former CBS News correspondnet Harry Smith. Smith subsequently produced a documentary about war correspondence, Dear Home: Letters From World War II, which aired on the History Channel in 1999. Six of the World War II letters collected by the Legacy Project were put on display that same year at the Smithsonian Institution's National Postal Museum. By the fall of 2002, the Legacy Project had received 70,000 letters, and some of them were reprinted in The New York Times.

In 1999, Carroll edited the book Letters of a Nation: A Collection of Extraordinary American Letters, a collection of more than 200 letters from famous and not-so-famous Americans from the past 350 years. It became a New York Times best-seller. Letters of a Nation was read by Senator Robert Torricelli, who later contacted Carroll and suggested that he compile a book of influential American speeches as well. A year later, Carroll and Torricelli collaborated on In Our Own Words: Extraordinary Speeches of the American Century, an edited volume of 150 important American addresses.

In 2001, Carroll edited the book War Letters: Extraordinary Correspondence from American Wars, based on the wartime correspondence collected by the Legacy Project. Scribner's publishers gave Carroll a $500,000 advance for the book (money Carroll donated to veterans' groups). The book became a New York Times best-seller, debuting at #13 on the best-selling nonfiction list. It rose as high as #10 two weeks later. Director Steven Spielberg heard about some of the letters, and several were included in Spielberg's short documentary The Unfinished Journey (which screened in front of the Lincoln Memorial on the National Mall in Washington, D.C., on December 31, 1999, as part of the millennial celebrations). The book was turned into "War Letters," a 2001 episode of television program American Experience directed by Robert Kenner. A play based on the book was produced in Los Angeles in 2002. The Legacy Project spawned a second collection of war letters, Behind the Lines: Powerful and Revealing American and Foreign War Letters...and One Man's Search to Find Them, in 2006.

In 2002, Carroll used a $50,000 corporate donation to print 100,000 copies of four new Armed Services Editions books given away for free to active-duty American military personnel serving in combat zones overseas.

In 2004, the National Endowment for the Arts (NEA) commissioned Carroll to edit a collection of military writings, which became the book Iraq, Afghanistan, and the Home Front, in the Words of U.S. Troops and Their Families. The NEA held 20 writing workshops at military bases around the nation, and Carroll edited the servicemembers' works into the new book. The book was published in May 2008. A second collection of poems, short stories, essays, and letters by military personnel who served in Iraq and Afghanistan, titled Above and Beyond, was announced in 2005 but has not been published as of June 2009.

Grace Under Fire: Letters of Faith in Times of War, another book of collected war letters which focused on the role of religion and spiritual belief in wartime, was edited by Carroll and published in March 2007.

In June 2009, Carroll founded the "Here Is Where" campaign in association with National Geographic Traveler. The project is an all-volunteer effort to photograph and document historic locations in the United States.

Since 1994, Andrew Carroll has also served on the board of directors of the Literacy Volunteers of America. He is also an active Advisory Board member of Carolina for Kibera, a charity founded by a U.S. Marine that prevents violence and provides basic health care in the Kibera slum of Nairobi, Kenya. He was named a "Person of the Week" by ABC's World News Tonight and received the Iona Senior Services President's Award, both in 1994.