Mathematical physics is still math. They follows the usual math convention: there are axioms and theorems. Mathematical physics isn't really physics, it's more like math that was inspired or somewhat related to physics. There are a lot of things in them that are not physically relevant, like toy model that is not a real theory of anything, but has some mathematically similar properties to what a real theory would look like. Or sometimes they used techniques inspired by physics to tackle a question interested by mathematician.