An early work by Andre Norton and Phyllis Miller. Young Adult.
A novel more in the occult vein than fantasy.
A family is temporarily broken up by financial circumstances, and the three children are sent to live with a peculiar aunt out in the rural Northeast. The natives of the town shun the new people, and tell them their family is cursed. Then people start talking to the children . . .DEAD people start talking to them . . .