Is it ok to give up?

A lot of  people say they don't like math, and I don't doubt them. They say it's not "their thing."

But I actually HATE math.
I don't even know if hate is a strong enough word. Math makes me fell ill, constantly inadequate, stressed, and powerless. I've struggled with it my entire life, it has evolved over the years. From elementary school, I have been set aside and pulled apart from the crowd to receive special help with math. Don't think for a second I didn't notice I was deficient in a way my classmates were not. Kids notice those sort of things as early as the age of four. A sense of being abnormal in an academic sort-of way started very young for me. In middle school I had to go to classes an hour before school started to receive extra help. I was not alone, however, it was a class of about fifteen kids. But I still felt isolated from my peers. Getting B's in math during middle school was a norm that I accepted. I normally got C's on tests and A's on daily assignments, all while taking that extra morning class and asking the teacher for help after school. As long as I was improving or maintaining an average, the extra work seemed worth it. I had to do all that just to try to stay at the same level as the kids in my class.
High school algebra hit me like a brick wall. I had an incredibly supportive teacher, but just could not grasp the material. Nightly homework was met with tears of frustration! I was constantly in my math class in the mornings, I got a solid C- the first semester..... At that point my parents stepped in, we had a meeting with my math teacher and councilor to see what could be done. It was like I was sliding down a dangerous slide, and we were trying to prevent it. I got a tutor, I was in there every morning, and still only pulled a C out of the class. Such grand efforts for little-to-no improvement frustrated me. I felt defeated, stupid, "less-than," ashamed, and confused. Feelings that have persisted throughout my high school career. I retook algebra my sophomore year, at the advice of the math department head. Now I was officially behind my peers. And that has been really hard to accept, I was convinced I was stupider than my peers. But I made it through that year with the help of another supportive teacher and tutor. I got a B+ over all and I was ecstatic about it! I went to lunch after receiving that grade, just to be met by some friends complaining to each other how stupid they were cause they got an A- in like, pre-calc! Now..... I know I'm not suppose to compare myself to others, heard that speech a thousand times.... But if you are wondering where I got the idea that I was somehow less than, and somehow stupid, there that was to reaffirm it.
Going into my junior year I did not have a supportive teacher, which was like adding ten pounds to the already massive load I was carrying into math. I was in Geometry, and struggled very much. I had more emotional break downs, with even less support. I got another tutor and was receiving help from the math department head in the mornings, but the way the class was set up, if you bombed a test you could spend the next few weeks clawing your way back up, just to be nocked down by the next test. I was flat-linning that class, constantly getting a solid D. Which put even more pressure on the college aspect of things. I ended the year with a D+.
I hit a low I had never come to before. I couldn't afford to continue to spiral downward. Over the summer I even went to a professional psychologist to see if I had anything mentally wrong with me. Nothing significant came up in the least. I have a tendency to switch numbers sometimes, but nothing official. I mostly have what I describe as a "big black wall" where when I'm testing I can't retrieve the material I studied! When testing I become overwhelmed and start making up mathematical rules just to complete the test. I've tried about four different types of testing scenarios during my time in high school as well, and none have yielded positive results!
I became convinced that the effort I was putting in to learning math was all for nothing, if I just keep continuing to go downward. Currently I am in Algebra 2 and am on track to receive another D+. I had to hold back frustrated tears today as I got my horrifying test back.
How can all this effort not do anything!? If anything I have gone DOWN in mathematical achievement. After six years of trying to make it work, and watching it not, it makes me want to give up. I'm to the point where every single morning I contemplate going to my math class. Spending my time surrounded by kids who understand the material and who still have a chance to improve in math just sounds like hell. Yet I go in there five days out of the week.
I have been writing in past-tense for most of this post, "I thought I was stupid" or "I felt somehow lesser." But those feelings not only happened in the past, they are happening now. I still feel stupid, I still feel lesser than my classmates. I feel like I slid down that slide and have no chance to climb back up. I know I'm not all around stupid. I shine in my social studies class, I receive A's in english, and I don't suck at science. But in my mental self-punishing pursuit of perfection, if I haven't succeeded in one thing, that failure ultimately overshadows the other successes. And I am well versed psychology, so I know how ridiculous that sounds, I know logically that my self-worth and my success is not measured by math alone. But some part of my brain won't let it go.
Math just causes me unnecessary pain, problems, stress, tears, anxiety, pressure, and unimaginable amounts of frustration. If anything outside of school were causing those feelings I would stop it immediately! So why have I given math 11 years!? Is it ok to give it up yet!? Cause I don't have much more to give!