Well, first things first. For the physics understanding of manifolds, you don't need to know everything about topological manifolds.

It is sufficient to just look at an introductory course on differential geometry, take a look at Riemannian Geometry and then Lorentzian geometry. That's assuming you want manifolds for relativity purposes.

For Hamiltonian systems it is interedting to learn about symplectic manifolds, but again, for physics purposes you do not need to understand or know every topological property of manifolds. You just need to know what charts are and what differential forms are. But, again, gor a physics point of view you do not need to know why you can define those. You just need to know you can and what they do.

That being said, I can only applaud you if you do want to study topology at a base level such that you know the "why" parts. Personally, my professors always wrote their own lecture books in LaTeX, so I'm not sure which books are good for entry level topology and differential geometry, so I can't attest for the books that others have linked here. I only used more advanced books for my Master Thesis.