Mathematics for AI & ML
Linear Algebra
- Scalars, Vectors, and Matrices
- Vector Operations
- Matrix Operations
- Special Matrices
- Rank, Determinant, and Inverses
- Eigenvalues and Eigenvectors
- Singular Value Decomposition (SVD)
- Positive Semi-Definite Matrices & Covariance
- Linear Transformations
- Subspace Basis
- Linear Independence & Orthogonality
- Projections & Least Squares
- Matrix Factorizations in ML (LU, QR, Cholesky)
- Pseudo-Inverse & Ill-Conditioned Systems
- Block Matrices & Kronecker Products
- Spectral Decomposition & Applications
Calculus & Optimization
- Higher-Order Derivatives: Hessian & Curvature
- Convexity & Optimization Landscapes
- Gradient Descent & Variants
- Advanced Optimization (Momentum, Adam, RMSProp)
- Constrained Optimization (Lagrange, KKT)
- Integration Basics
- Continuous Distributions & Calculus of Probability
- Differential Equations in ML (Neural ODEs)
- Taylor Series & Function Approximations
- Multivariable Taylor Expansions
- Integral Transforms (Laplace, Fourier)
- Measure Theory Lite
Probability
- Why Probability in ML?
- Random Variables & Distributions
- Expectation, Variance & Covariance
- Conditional Probability & Bayes Theorem
- Independence & Correlation
- LLN & Central Limit Theorem
- Maximum Likelihood Estimation (MLE)
- Maximum A Posteriori (MAP)
- Entropy, Cross-Entropy & KL Divergence
- Markov Chains
- Bayesian Inference
Statistics
- Data Summaries: Mean, Median, Mode, Variance
- Common Distributions (Normal, Binomial, Poisson)
- Correlation & Covariance
- Sampling & Sampling Distributions
- Estimation & Confidence Intervals
- Hypothesis Testing
- ANOVA
- Resampling: Bootstrap & Permutation Tests
- MLE vs Method of Moments
- Bayesian Statistics in Practice
- Bias, Variance & Error Decomposition
- Cross-Validation
- Statistical Significance in ML
- Nonparametric Statistics
- Multivariate Statistics
- Time Series Basics
- Causal Inference
- Experimental Design & A/B Testing
Miscellaneous Mathematics
- Iterative Solvers: CG, Power Method, Lanczos
- Numerical Stability & Conditioning
- Sparse Matrices & Efficient Computation
- Markov, Chebyshev, Hoeffding Inequalities
- Jensen’s Inequality & Convex Functions
- Generalization Bounds in ML
- Fisher Information Matrix
- Natural Gradient Descent
- Information Geometry in Variational Inference
- Martingales & Random Walks
- Brownian Motion & Stochastic Differential Equations
- Applications in RL & Diffusion Models
- Curse of Dimensionality
- Johnson–Lindenstrauss Lemma
- High-Dimensional Statistics
- Tensors & Tensor Operations
- Manifold Learning
- Spectral Methods in ML