Humans are actually very poor multitaskers and the ability to carry out cognitively complex tasks like arithmetic will fall very fast if you are trying to do multiple things at the same time or are in a high stress situation.
You don't have to be an Einstein to do your job effectively and this is hardly proof that you are poor at maths (which covers so much more than arithmetic). I'd suggest that you shouldn't lose sleep over this, keep a small calculator handy at work, I'm sure your employers won't care as long as you do the work right.