Rank, Determinant, and Inverses

Understanding the rank, determinant, and inverse of a matrix helps us reason about when systems of equations are solvable, when data is redundant, and when certain machine learning methods can be applied.

Rank

The rank of a matrix is the maximum number of linearly independent rows or columns.

Rank tells us how much useful information a matrix has.
If rank = number of columns, the columns are linearly independent.
If rank < number of columns, some features are redundant (linear combinations of others).

Mini Example:

A = [\begin{matrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{matrix}]

Here, rank $(A) = 2$ because one row/column is a linear combination of the others.

Explanation of Rank

Rank tells us whether the matrix is “full information” or if it contains redundancy.

In ML: rank-deficient design matrices $X$ cause issues in regression because $X^{T} X$ becomes non-invertible.

Determinant

The determinant of a square matrix is a scalar value that summarizes important properties:

If det $(A) = 0$ , the matrix is singular (non-invertible).
If det $(A) \neq 0$ , the matrix is invertible.
Geometric meaning: determinant is the scaling factor of volume under the transformation.

Mini Example:

A = [\begin{matrix} 4 & 7 \\ 2 & 6 \end{matrix}]

det $(A) = 4 \cdot 6 - 7 \cdot 2 = 10$

Explanation of Determinant

Determinant tells us if a matrix “collapses” space.

det = 0 → volume collapses to lower dimension (no inverse).
det ≠ 0 → transformation preserves space (matrix invertible).

Inverse (Recap)

The inverse $A^{- 1}$ satisfies:

A A^{- 1} = A^{- 1} A = I

Only square, non-singular matrices (det ≠ 0) have inverses.
In ML: used in the normal equation for regression, but avoided in practice.

Mini Example:
For the same $A$ above:

A^{- 1} = \frac{1}{10} [\begin{matrix} 6 & - 7 \\ - 2 & 4 \end{matrix}]

Hands-on with Python and Rust

PythonRust

python

import numpy as np

A = np.array([[4, 7], [2, 6]])
B = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# Rank
rank_B = np.linalg.matrix_rank(B)

# Determinant
det_A = np.linalg.det(A)

# Inverse
inv_A = np.linalg.inv(A)

print("Rank of B:", rank_B)
print("Determinant of A:", det_A)
print("Inverse of A:\n", inv_A)

rust

use ndarray::array;
use ndarray::Array2;
use ndarray_linalg::{Determinant, Inverse, Rank};

fn main() {
    let a: Array2<f64> = array![[4.0, 7.0], [2.0, 6.0]];
    let b: Array2<f64> = array![
        [1.0, 2.0, 3.0],
        [4.0, 5.0, 6.0],
        [7.0, 8.0, 9.0]
    ];

    // Rank
    let rank_b = b.rank().unwrap();

    // Determinant
    let det_a = a.det().unwrap();

    // Inverse
    let inv_a = a.inv().unwrap();

    println!("Rank of B: {}", rank_b);
    println!("Determinant of A: {}", det_a);
    println!("Inverse of A:\n{:?}", inv_a);
}

Connection to ML

Rank → tells us if features are redundant (multicollinearity).
Determinant → indicates whether transformations preserve volume (non-singular).
Inverse → appears in closed-form regression solutions (but we prefer iterative solvers).

Next Steps

Continue to Eigenvalues and Eigenvectors.

Rank, Determinant, and Inverses ​

Rank ​

Determinant ​

Inverse (Recap) ​

Hands-on with Python and Rust ​

Connection to ML ​

Next Steps ​