I mean, no one (as far as I can tell) has been interpreting that to mean “a person who can’t speak”, or someone mentally disabled (in a known way, or in a way that does not create harm or danger to others).
Context really needs to be a strong determining factor. Dictionary definitions or etymology can help sometimes, but current, present-day usage and the actual target and meaning in the sentence in question has to be important. Otherwise you can end up in situations where you can’t label anything as “bigoted” because actual bigots will get upset at how they don’t actually subscribe to that very specific way of thinking…
A commonly used term these days used to be significantly dangerous to be labeled as such. Now no one thinks about what it used to mean to be called a ‘bastard’. I think many of these words are the same: so far removed from their original meaning as to be context-less at this point.