you can do it in any order you like, since these properties hold true:
∀a,b∈ℤ, ∀n∈ℕ\{0},
(a mod n)+(b mod n) = a+b mod n
(a mod n)×(b mod n) = a×b mod n
I usually find it easier to convert the whole matrix to F5 and just do the whole arithmetic in F5, it's pretty easy since these are finite fields with very few elements. Once you start going into stuff like F37 or things like that, it might be useful to carry the arithmetics as if you were in ℝ (always keeping the fractions, never going to decimal form) and convert everything to F37 at the end.
As I'm sure you already know, remember that Fn is a field if and only if n is a prime number, so you can't do stuff like gaussian elimination if n isn't a prime number.