Secularists say the problem with organized religion is the fact that it enables and justifies bad actions. Organized religion may do a lot of good in the world, but it also feeds into people's worst behaviors.
A lot of the attacks brought against organized religion are not unique to organized religion. Sure, people do bad things in the name of religion, but people also do bad things in the name of politics or sports or money. Here, /u/BlindWillieJohnson replies to a post on the /r/ChangeMyView subreddit, which argues society should move past religion.
TL;DR: Organized religion is not uniquely bad—it's just humans.