“If they were really an abuser, their ex partners would have said they were too.”
Abusers do not abuse everyone. Abusers can be selective and find victims that they think are easier control (usually with systemic indifferences; age, money, wealth).
A lot of people don’t believe the victims of abuse because they have only had perfectly pleasant interactions with the abuser. This is a dangerous trap to fall into, and it’s important to remember that abusers will never abuse everyone. They need to be selective of their victims. That’s because if they abuse everyone they come across, no one will get close enough to them for them to manipulate. Thus they”ll have nobody to control and enact their abuse upon.