I'm not good in Math, and I don't think I actually "hate" it.
But I think the most difficult thing in Math was to learn it in a classroom setting. Being in a classroom means having to adjust to whatever speed the class adapts when learning lessons - and because we're supposed to follow a calendar of lessons, this means we need to move on to another topic before actually mastering the last. This means everyone has to adjust to the others that got the topic already.
I think the thing I don't like about Math is that everything has an answer that cannot be refuted. I'm more on the, "you either memorize the thing or you explain it in your own terms" - but when it comes to definitive solutions, I find them hard to find because everything has a process, and each process leads to only one answer. I find it hard to adjust in an environment where everything is narrow and constricted.