Is 4i a complex number?

4i is indeed a complex number. As is 3. It is perfectly acceptable for the real part or the imaginary part to be 0.

Edit: To be very pedantic, there may be situations where one distinguishes the real number x and the complex number x+0i. However, this subset of the complex numbers with zero imaginary part behaves exactly like the real numbers, so they are often identified with them.
Yep!

4i can be re-written as 0+4i, like you said.

That said 3 (which can be written as 3+0i) is *also* a complex number!

Complex numbers are a superset of the real numbers, meaning that all real numbers are complex numbers. This is similar to how all integers are rational numbers, or how all rational numbers are real numbers. Whether you see 3 as a real number or a complex number is entirely context-dependent, and either option is correct.
Every real number is also a complex number (reals are subset of complex numbers) same as every integer is a real number (you don't need to write 1.0 for 1 to be considered real number), so yes 3 is a complex number no matter if you write is as just "3" or "3 + 0i". Same with imaginary numbers - they are also a subset of complex numbers so "4i" is complex.
Every number is a complex number. Since a complex number is written in the form "a+bi", a and b can either (or both) be 0, which can lead to a, bi, or just 0.
0 is a real number, so a and b can be 0, and the number is still complex.
Think of it from a venn diagram viewpoint. The whole set of real numbers is a subset of complex numbers. Every real number is a complex number but not every complex number is a real number.
3 is a natural number, an integer, a rational number, a real number and a complex.

3 can be considered as well a quaternion where a, b & c are all equal to 0

Or a one by one matrix (3)