“If they really abused that victim, other people would say the same.”
Again, abusers do not abuse everyone. Likewise, if someone that accuses someone is met with a smear campaign, it’s likely this would scare other victims into silence.
It remains a high-stakes endeavour to accuse a powerful man of sexual misconduct in public life. It still seems to be a numbers game. One woman’s word alone is rarely believed (Brett Kavanaugh was unscathed); 50 might do it (it took 87 to bring down Harvey Weinstein); 150 is ideal (the number of women and girls who accused Larry Nassar, the US gymnasts’ doctor).