She is the daughter of historian and historical biographer Lady Antonia Fraser and the late Sir Hugh Fraser, a British Conservative politician. Her stepfather was the playwright Harold Pinter, the 2005 Nobel Laureate in Literature, her mother's second husband until his death in 2008. Her maternal grandparents were the late Elizabeth Longford, also an eminent biographer, and the late Lord Longford, a well-known politician, social reformer, and author.

She was named after Scottish Jacobite Flora MacDonald. Using her maiden name Flora Fraser, she has written biographies of Emma Hamilton, Caroline of Brunswick, the daughters of George III, and Pauline Bonaparte.

When she was 21, Fraser married Robert Powell-Jones with whom she had a daughter. Powell-Jones died of a heart attack in 1998, aged 44. She later married Peter Soros, a nephew of American currency speculator and philanthropist George Soros, with whom she has two children. Fraser and Soros separated in 2009. Fraser currently lives in London.