You can’t control the negative thoughts someone might have about you, but you can try not to deserve them.
“What delight comes to the one who follows God’s ways! He won’t walk in step with the wicked, nor share the sinner’s way, nor be found sitting in the scorner’s seat.” — Psalm 1:1, The Passion Translation
“And they come to you as people come, and they sit before you as my people, and they hear what you say but they will not do it; for with lustful talk in their mouths they act; their heart is set on their gain. And behold, you are to them like one who sings lustful songs with a beautiful voice and plays well on an instrument, for they hear what you say, but they will not do it.” — Ezekiel 33:31-32, ESV