Baldwin himself put it this way: “What this does to the subjugated is to destroy his sense of reality.”
Notice that he does not say that reality itself is destroyed, but only the “sense” of reality. You cannot take from a person their dignity, though you can treat them so cruelly, and so unjustly, that you can make them believe that their dignity is gone, or that it was never there to begin with.