"But in recent months, Pepe’s been almost entirely co-opted by the white supremacists who call themselves the 'alt-right.' They’ve decided to take back Pepe by adding swastikas and other symbols of anti-semitism and white supremacy."
Pepe has been consciously used by a group of people to promote deplorable ideas. That includes one meme depicting a Nazi soldier handing out a Jewish star saying, "You get a gold star." Another meme featured a semitic-looking Pepe smiling as a plane crashes into the World Trade Center. The connection of Pepe to Trump's politics on immigration and Muslims is also pretty direct.
Memes are interpretable. They are what you want them to be. Do you want Pepe to be a janitor? Poof, he's a janitor. Do you want Pepe to be Superman? Just make it so.
Maybe some bad people have taken to Pepe, that doesn't change the nature of Pepe himself. It doesn't make others using him de facto racists.