As a start, I suggest learning the following:
\- probability (distributions, basic manipulations)
\- statistical summaries (univariate and bivariate)
\- hypothesis testing / confidence intervals
\- linear regression
\- basic understanding of arranging data in vectors and matrices
\- operators (matrix multiplication, ...)
\- basic differentiation and integration (at least of polynomials)
Information Theory (Discrete):
\- entropy, joint entropy, conditional entropy, mutual information
For statistics, I highly recommend:
"Practice of Business Statistics"
by David S. Moore, George P. McCabe, William M. Duckworth and Stanley L. Sclove
To learn about machine learning, I recommend both of these:
"Computer Systems That Learn"
by Weiss and Kulikowski
"Data Mining: Practical Machine Learning Tools and Techniques"
by Ian H. Witten, Eibe Frank, Mark A. Hall and Christopher J. Pal
The 4th edition (2016) has ISBN-13: 978-0128042915, though older editions are fine and likely less expensive.