One unfortunate trend of history is the tendency of oppressor/victim categories (along with favored groups) to flip flop over time as social groups react and counter-react to each other. It never solves anything and the blame game needs to ultimately be transcended if real tolerance and equality is to be possible.
It's one reason I dislike the idea of labeling myself as a progressive. I can't stand the tendency some progressives have to put an icing of nice sounding rhetoric about tolerance and equality over a passive-aggressive "blame men/caucasians" mentality. It would be far better if progressives went back to the goals of empowering workers they had in the 1930s and 19th century.