Analyzing the Limits of Self-Supervision in Handling Bias in Language