Math is made up and none of it is physical. No need for units. Vectors are mathematical. So not physical. No need for units.
Physics uses math to represent physical things. Physical things are measured in units.
But enough philosophy. Really, you use units when they are useful, and not when they are not.