I'm not into math, but I love drawing and I'm going through the same thing as you. I've never tried learning anything deeply like I have done with draftsmanship before. Through most of our life, it's easy to learn the basics of many subjects. To master it, however, is another story. My theory is that we're forcing our minds to expand beyond limits that it has never approached before. Therefore, the pain occurs because our brains are physically changing.
But I'm no neuroscientist, I'm just trying to rationalize this learning pain.