In 1971, Remy was drafted in the 8th round (129th overall) by the California Angels. He played four seasons in their minor league system before making his major league debut in 1975. Remy played three seasons for the Angels as their starting second baseman before being traded to the Boston Red Sox after the 1977 season.

Remy continued as the Red Sox starter at second in 1978, being selected to play in the All-Star Game, in which he did not appear. He continued as their starting second basemen for the next six seasons, although he was often hampered by injuries. A knee injury in May 1985 led to Remy's release by the Red Sox in December 1985 and to his retiring during spring training a few months later in 1986. Bill James, in his Historical Abstract rated him as the 100th greatest second baseman of all time as of 2002.

Since 1988, Remy has found success in broadcasting, working for the New England Sports Network , as the color commentator for all NESN Red Sox broadcasts. Since 2001, Remy has been teamed with play-by-play announcer Don Orsillo. NESN celebrated Jerry Remy Day on June 24, 2008, in honor of Remy's 20 years of service for the network. He is currently serving as the first president of Red Sox Nation.

Remy owns a hot dog stand, "RemDawg's," located on Yawkey Way outside of Fenway Park in Boston, as well as Jerry Remy's Sports Bar & Grill in Logan International Airport since 2008 and Jerry Remy's Sports Bar & Grill behind Fenway Park on Boylston Street, that opened March 9, 2010 to great success. The food was highly acclaimed by staff and patrons alike. He also runs a web site, The Remy Report, and is the author of the book Watching Baseball (co-authored with Corey Sandler) and the children's books Hello Wally and Wally the Green Monster and His Journey Through Red Sox Nation!.

Jerry Remy was inducted into the Boston Red Sox Hall of Fame in 2006.

In November 2008, Remy had surgery to remove a "very small, low-grade cancerous area" from his lung, most likely a result of years of smoking cigarettes. During his recovery from the surgery, he suffered from an infection as well as a bout of pneumonia. Due to fatigue and depression, Remy took an indefinite leave of absence from his broadcast duties for NESN, starting April 30, 2009.

On August 12, 2009, Remy went to Fenway Park and attended Red Sox manager Terry Francona's pre game press conference. He told both NESN and the Boston Globe that he had every intention of returning to broadcasting Red Sox games during the remainder of the 2009 season. He entered the NESN's broadcast booth during the top of the second inning during the night's game to speak with broadcasters Don Orsillo and Dennis Eckersley. It was the first time he had been in the booth since he took his leave of absence in April. In between the top and the bottom of the second inning, Remy, still in the booth, was shown on Fenway's center field scoreboard display, to which he received a standing ovation from the crowd attending the game. He revealed during the visit that he had suffered from depression following his physical problems of 2008 and that he was receiving therapy. On August 19, 2009, Remy released a statement announcing his return to commentating on August 21, 2009 against the New York Yankees. Remy stated that he would likely skip some road trips. He has since returned full time for the 2010 baseball season.