The metaphor I've seen is that being non-white is like climbing an escalator that's going down. If you, personally, aren't doing anything to speed the escalator up that's great - but there are plenty of people who are, and it might be good to do something to slow it down.