Answer:
It is human nature.
Explanation: After living a certain way, in this case, the victims/survivors of slavery, it was all that they knew. Their way of life was built around it. Their family, friends, homes and even "work" connections were there. There is a level of comfort that comes from not leaving all that we know.
Many of the former slaves did leave eventually. I would suggest that family is so important and those that did not, wanted to stay near them.